Robotic system with coordination mechanism and methods of operating the same

ABSTRACT

A system and method for operating a robotic system to coordinate and integrate multiple tasks for performing operations is disclosed. The robotic system may identify a set of tasks associated with a triggered operation. Accordingly, the robotic system may coordinate and control actions across subsystems, robotic units, task stations, or a combination thereof to sequentially perform the set of tasks and complete the operation.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of U.S. patent application Ser. No.16/740,251, filed Jan. 10, 2020, now issued as U.S. Pat. No. ______,which claims the benefit of U.S. Provisional Patent Application No.62/792,348, filed Jan. 14, 2019, both of which are incorporated byreference herein in their entirety.

This application contains subject matter related to U.S. patentapplication Ser. No. 16/739,971, filed Jan. 10, 2020, titled “CONTROLLERAND CONTROL METHOD FOR ROBOT SYSTEM,” which is incorporated herein byreference in its entirety.

TECHNICAL FIELD

The present technology is directed generally to robotic systems and,more specifically, to systems, processes, and techniques forcoordinating operations of multiple units.

BACKGROUND

With their ever-increasing performance and lowering cost, many robots(e.g., machines configured to automatically/autonomously executephysical actions) are now extensively used in many fields. Robots, forexample, can be used to execute various tasks (e.g., manipulate ortransfer an object through space) for manufacturing and/or assembly,packing and/or packaging, transport and/or shipping, etc. In executingthe tasks, the robots can replicate human actions, thereby replacing orreducing human involvements that are otherwise required to performdangerous or repetitive tasks.

However, despite the technological advancements, robots often lack thesophistication necessary to duplicate human interactions required forexecuting larger and/or more complex tasks. For example, robot-to-robotinteractions often require human intervention to fully coordinate andcombine a sequence of tasks. Accordingly, there remains a need forimproved techniques and systems for managing operations and/orinteractions between robots.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of an example environment in which a roboticsystem with a coordination mechanism may operate.

FIG. 2 is a block diagram illustrating the robotic system in accordancewith one or more embodiments of the present technology.

FIG. 3 is an illustration of example task units associated with therobotic system of FIG. 1 in accordance with one or more embodiments ofthe present technology.

FIG. 4 is an illustration of an example control diagram for the roboticsystem of FIG. 1 in accordance with one or more embodiments of thepresent technology.

FIG. 5A is an illustration of a first example task station in accordancewith one or more embodiments of the present technology.

FIG. 5B is a flow diagram for a method of operating the robotic systemof FIG. 1 in accordance with one or more embodiments of the presenttechnology.

FIG. 6A is an illustration of a second example task station inaccordance with one or more embodiments of the present technology.

FIG. 6B is a flow diagram for a method of operating the robotic systemof FIG. 1 in accordance with one or more embodiments of the presenttechnology.

FIG. 7A is an illustration of a third example task station in accordancewith one or more embodiments of the present technology.

FIG. 7B is a flow diagram for a method of operating the robotic systemof FIG. 1 in accordance with one or more embodiments of the presenttechnology.

FIG. 8A is an illustration of a fourth example task station inaccordance with one or more embodiments of the present technology.

FIG. 8B is a flow diagram for a method of operating the robotic systemof FIG. 1 in accordance with one or more embodiments of the presenttechnology.

FIGS. 9A and 9B are illustrations of example task transitions inaccordance with one or more embodiments of the present technology.

FIG. 9C is an illustration of example transport units in accordance withone or more embodiments of the present technology.

FIG. 10 is a flow diagram for a method of operating the robotic systemof FIG. 1 in accordance with one or more embodiments of the presenttechnology.

DETAILED DESCRIPTION

Systems and methods for robotic systems with automated objectdetection/registration mechanisms are described herein. A robotic system(e.g., an integrated system of devices that executes one or moredesignated tasks) configured in accordance with some embodimentsautonomously executes sequences of integrated tasks (e.g., operations toachieve corresponding goals) by coordinating operations of multipleunits (e.g., robots).

The integrated tasks or operations can include receiving operations,stocking operations, shipping operations, and/or other operations. Thereceiving operation can include a sequence of tasks for receivingincoming shipments of objects (e.g., packages and/or boxes includingitems). The stocking operation can include a sequence of tasks forplacing the received objects and/or items in storage locations. Thestocking operation can further include a sequence of tasks forreorganizing or regrouping objects and/or items for storage. Theshipping operation can include a sequence of tasks for groupingitems/objects for outbound shipments. As described in detail below, thesequenced tasks can include devanning tasks, sorting tasks, storagegrouping tasks, group manipulation tasks, package opening tasks, rackingtasks, picking tasks, packing tasks, and/or outbound grouping tasks.Also, as described below, the robotic system can coordinate interactionsbetween multiple corresponding units, systems, and/or stations toperform the operations.

Traditional operations require inputs or assistance from human operatorsin executing typical integrated tasks. Traditional systems lack thesophisticated interaction between multiple robots and require operatorassistance in connecting an end of a task of one robot with a beginningof a task for a different robot. For example, traditional systems may beable to access the bins corresponding to an order but require humanoperators to group/sequence the ordered items for the order. Also, forexample, traditional systems may include picker robots that operateaccording to fixed inputs/outputs (e.g., conveyor inputs/outputs), butlack the sophistication to interact with other units to vary theinputs/outputs.

In comparison, the robotic system disclosed herein coordinates andcontrols the interactions between separate robotic units and/or stationsto execute the operations, thereby reducing or eliminating humanassistance for the execution. For example, the robotic system canidentify operating zones, operating paths, transition locations,movement plans, corresponding timings, or a combination thereof for eachof the units. Also, the robotic system can include one or morealgorithms for sequencing the tasks of the different units and/or one ormore protocols for controlling interactions between the units. Therobotic system can further account for the interaction between multipleunits and coordinate storage of items according to accessibility,projected load/order, estimated throughput, or a combination thereof.Details of the coordination and the controls are described below.

In the following description, numerous specific details are set forth toprovide a thorough understanding of the presently disclosed technology.In other embodiments, the techniques introduced here can be practicedwithout these specific details. In other instances, well-known features,such as specific functions or routines, are not described in detail inorder to avoid unnecessarily obscuring the present disclosure.References in this description to “an embodiment,” “one embodiment,” orthe like mean that a particular feature, structure, material, orcharacteristic being described is included in at least one embodiment ofthe present disclosure. Thus, the appearances of such phrases in thisspecification do not necessarily all refer to the same embodiment. Onthe other hand, such references are not necessarily mutually exclusiveeither. Furthermore, the particular features, structures, materials, orcharacteristics can be combined in any suitable manner in one or moreembodiments. It is to be understood that the various embodiments shownin the figures are merely illustrative representations and are notnecessarily drawn to scale.

Several details describing structures or processes that are well-knownand often associated with robotic systems and subsystems, but that canunnecessarily obscure some significant aspects of the disclosedtechniques, are not set forth in the following description for purposesof clarity. Moreover, although the following disclosure sets forthseveral embodiments of different aspects of the present technology,several other embodiments can have different configurations or differentcomponents than those described in this section. Accordingly, thedisclosed techniques can have other embodiments with additional elementsor without several of the elements described below.

Many embodiments or aspects of the present disclosure described belowcan take the form of computer- or processor-executable instructions,including routines executed by a programmable computer or processor.Those skilled in the relevant art will appreciate that the disclosedtechniques can be practiced on computer or processor systems other thanthose shown and described below. The techniques described herein can beembodied in a special-purpose computer or data processor that isspecifically programmed, configured, or constructed to execute one ormore of the computer-executable instructions described below.Accordingly, the terms “computer” and “processor” as generally usedherein refer to any data processor and can include Internet appliancesand handheld devices (including palm-top computers, wearable computers,cellular or mobile phones, multi-processor systems, processor-based orprogrammable consumer electronics, network computers, mini computers,and the like). Information handled by these computers and processors canbe presented at any suitable display medium, including a liquid crystaldisplay (LCD). Instructions for executing computer- orprocessor-executable tasks can be stored in or on any suitablecomputer-readable medium, including hardware, firmware, or a combinationof hardware and firmware. Instructions can be contained in any suitablememory device, including, for example, a flash drive and/or othersuitable medium.

The terms “coupled” and “connected,” along with their derivatives, canbe used herein to describe structural relationships between components.It should be understood that these terms are not intended as synonymsfor each other. Rather, in particular embodiments, “connected” can beused to indicate that two or more elements are in direct contact witheach other. Unless otherwise made apparent in the context, the term“coupled” can be used to indicate that two or more elements are ineither direct or indirect (with other intervening elements between them)contact with each other, or that the two or more elements cooperate orinteract with each other (e.g., as in a cause-and-effect relationship,such as for signal transmission/reception or for function calls), orboth.

Suitable Environments

FIG. 1 is an illustration of an example environment in which a roboticsystem 100 with a coordination mechanism may operate. The robotic system100 can include and/or communicate with one or more units (e.g., robots)configured to execute one or more tasks. Aspects of the coordinationmechanism can be practiced or implemented by the various units.

For the example illustrated in FIG. 1, the robotic system 100 caninclude an unloading unit 102, a transfer unit 104 (e.g., a palletizingrobot and/or a piece-picker robot), a transport unit 106, a loading unit108, or a combination thereof in a warehouse or a distribution/shippinghub. Each of the units in the robotic system 100 can be configured toexecute one or more tasks. For another example, the task can includeplacing the objects on a target location (e.g., on top of a palletand/or inside a bin/cage/box/case). The robotic system can derive plans(e.g., placement locations/orientations, sequence for transferring theobjects, and/or corresponding motion plans) for placing and/or stackingthe objects. Each of the units can be configured to execute a sequenceof actions (by, e.g., operating one or more components therein)according to one or more of the derived plans to execute a task.

In some embodiments, the task can include manipulation (e.g., movingand/or reorienting) of a target object 112 (e.g., one of the packages,boxes, cases, cages, pallets, etc. corresponding to the executing task)from a start location 114 to a task location 116. For example, theunloading unit 102 (e.g., a devanning robot) can be configured totransfer the target object 112 from a location in a carrier (e.g., atruck) to a location on a conveyor belt. Also, the transfer unit 104 canbe configured to transfer the target object 112 from one location (e.g.,the conveyor belt, a pallet, or a bin) to another location (e.g., apallet, a bin, etc.). For another example, the transfer unit 104 (e.g.,a palletizing robot) can be configured to transfer the target object 112from a source location (e.g., a pallet, a pickup area, and/or aconveyor) to a destination pallet. In completing the operation, thetransport unit 106 can transfer the target object 112 from an areaassociated with the transfer unit 104 to an area associated with theloading unit 108, and the loading unit 108 can transfer the targetobject 112 (by, e.g., moving the pallet carrying the target object 112)from the transfer unit 104 to a storage location (e.g., a location onthe shelves).

The robotic system 100 can combine and/or sequence tasks to perform anoperation that achieves a goal, such as to unload objects from a truckor a van and store them in a warehouse or to unload objects from storagelocations and prepare them for shipping. Details regarding the operationand the associated actions are described below.

For illustrative purposes, the robotic system 100 is described in thecontext of a shipping center; however, it is understood that the roboticsystem 100 can be configured to execute tasks/operations in otherenvironments/for other purposes, such as for manufacturing, assembly,packaging, healthcare, and/or other types of automation. It is alsounderstood that the robotic system 100 can include other units, such asmanipulators, service robots, modular robots, etc., not shown in FIG. 1.For example, in some embodiments, the robotic system 100 can include adepalletizing unit for transferring the objects from cage carts orpallets onto conveyors or other pallets, a container-switching unit fortransferring the objects from one container to another, a packaging unitfor wrapping the objects, a sorting unit for grouping objects accordingto one or more characteristics thereof, a piece-picking unit formanipulating (e.g., for sorting, grouping, and/or transferring) theobjects differently according to one or more characteristics thereof, ora combination thereof.

The robotic system 100 and/or the units thereof can include physical orstructural members (e.g., robotic manipulator arms) that are connectedat joints for motion (e.g., rotational and/or translationaldisplacements). The structural members and the joints can form a kineticchain configured to manipulate an end-effector (e.g., the gripper)configured to execute one or more tasks (e.g., gripping, spinning,welding, etc.) depending on the use/operation of the robotic system 100.The robotic system 100 can include the actuation devices (e.g., motors,actuators, wires, artificial muscles, electroactive polymers, etc.)configured to drive or manipulate (e.g., displace and/or reorient) thestructural members about or at a corresponding joint. In someembodiments, the robotic system 100 can include transport motorsconfigured to transport the corresponding units/chassis from place toplace.

The robotic system 100 can include sensors configured to obtaininformation used to implement the tasks, such as for manipulating thestructural members and/or for transporting the robotic units. Thesensors can include devices configured to detect or measure one or morephysical properties of the robotic system 100 (e.g., a state, acondition, and/or a location of one or more structural members/jointsthereof) and/or of a surrounding environment. Some examples of thesensors can include accelerometers, gyroscopes, force sensors, straingauges, tactile sensors, torque sensors, position encoders, etc.

In some embodiments, for example, the sensors can include one or moreimaging devices (e.g., visual and/or infrared cameras, 2D and/or 3Dimaging cameras, distance measuring devices such as lidars or radars,etc.) configured to detect the surrounding environment. The imagingdevices can generate representations of the detected environment, suchas digital images and/or point clouds, that may be processed viamachine/computer vision (e.g., for automatic inspection, robot guidance,or other robotic applications). As described in further detail below,the robotic system 100 can process the digital image and/or the pointcloud to identify the target object 112, the start location 114, thetask location 116, a pose of the target object 112, a confidence measureregarding the start location 114 and/or the pose, or a combinationthereof.

For manipulating the target object 112, the robotic system 100 cancapture and analyze an image of a designated area (e.g., a pickuplocation, such as inside the truck or on the conveyor belt) to identifythe target object 112 and the start location 114 thereof. Similarly, therobotic system 100 can capture and analyze an image of anotherdesignated area (e.g., a drop location for placing objects on theconveyor, a location for placing objects inside the container, or alocation on the pallet for stacking purposes) to identify the tasklocation 116. For example, the imaging devices can include one or morecameras configured to generate images of the pickup area and/or one ormore cameras configured to generate images of the task area (e.g., droparea). Based on the captured images, the robotic system 100 candetermine the start location 114, the task location 116, the associatedpose, the motion plan, and/or other processing result.

In some embodiments, for example, the sensors can include positionsensors (e.g., position encoders, potentiometers, etc.) configured todetect positions of structural members (e.g., the robotic arms and/orthe end-effectors) and/or corresponding joints of the robotic system100. The robotic system 100 can use the position sensors to tracklocations and/or orientations of the structural members and/or thejoints during execution of the task.

Suitable System

FIG. 2 is a block diagram illustrating the robotic system 100 inaccordance with one or more embodiments of the present technology. Insome embodiments, for example, the robotic system 100 (e.g., at one ormore of the units and/or robots described above) can includeelectronic/electrical devices, such as one or more processors 202, oneor more storage devices 204, one or more communication devices 206, oneor more input-output devices 208, one or more actuation devices 212, oneor more transport motors 214, one or more sensors 216, or a combinationthereof. The various devices can be coupled to each other via wireconnections and/or wireless connections. For example, the robotic system100 can include a bus, such as a system bus, a Peripheral ComponentInterconnect (PCI) bus or PCI-Express bus, a HyperTransport or industrystandard architecture (ISA) bus, a small computer system interface(SCSI) bus, a universal serial bus (USB), an IIC (I2C) bus, or anInstitute of Electrical and Electronics Engineers (IEEE) standard 1394bus (also referred to as “Firewire”). Also, for example, the roboticsystem 100 can include bridges, adapters, processors, or othersignal-related devices for providing the wire connections between thedevices. The wireless connections can be based on, for example, cellularcommunication protocols (e.g., 3G, 4G, LTE, 5G, etc.), wireless localarea network (LAN) protocols (e.g., wireless fidelity (WIFI)),peer-to-peer or device-to-device communication protocols (e.g.,Bluetooth, Near-Field communication (NFC), etc.), Internet of Things(IoT) protocols (e.g., NB-IoT, LTE-M, etc.), and/or other wirelesscommunication protocols.

The processors 202 can include data processors (e.g., central processingunits (CPUs), special-purpose computers, and/or onboard servers)configured to execute instructions (e.g. software instructions) storedon the storage devices 204 (e.g., computer memory). In some embodiments,the processors 202 can be included in a separate/stand-alone controllerthat is operably coupled to the other electronic/electrical devicesillustrated in FIG. 2 and/or the robotic units illustrated in FIG. 1.The processors 202 can implement the program instructions tocontrol/interface with other devices, thereby causing the robotic system100 to execute actions, tasks, and/or operations.

The storage devices 204 can include non-transitory computer-readablemediums having stored thereon program instructions (e.g., software).Some examples of the storage devices 204 can include volatile memory(e.g., cache and/or random-access memory (RAM)) and/or non-volatilememory (e.g., flash memory and/or magnetic disk drives). Other examplesof the storage devices 204 can include portable memory drives and/orcloud storage devices.

In some embodiments, the storage devices 204 can be used to furtherstore and provide access to processing results and/or predetermineddata/thresholds. For example, the storage devices 204 can store masterdata 252 that includes descriptions of objects (e.g., boxes, cases,and/or products) that may be manipulated by the robotic system 100. Inone or more embodiments, the master data 252 can include registrationdata 254 for each such object. The registration data 254 can include adimension, a shape (e.g., templates for potential poses and/orcomputer-generated models for recognizing the object in differentposes), a color scheme, an image, an identification information (e.g.,bar codes, quick response (QR) codes, logos, etc., and/or expectedlocations thereof), an expected weight, other physical/visualcharacteristics, or a combination thereof for the objects expected to bemanipulated by the robotic system 100. In some embodiments, the masterdata 252 can include manipulation-related information regarding theobjects, such as a center-of-mass (CoM) location or an estimate thereofon each of the objects, expected sensor measurements (e.g., for force,torque, pressure, and/or contact measurements) corresponding to one ormore actions/maneuvers, or a combination thereof.

The communication devices 206 can include circuits configured tocommunicate with external or remote devices via a network. For example,the communication devices 206 can include receivers, transmitters,modulators/demodulators (modems), signal detectors, signalencoders/decoders, connector ports, network cards, etc. Thecommunication devices 206 can be configured to send, receive, and/orprocess electrical signals according to one or more communicationprotocols (e.g., the Internet Protocol (IP), wireless communicationprotocols, etc.). In some embodiments, the robotic system 100 can usethe communication devices 206 to exchange information between units ofthe robotic system 100 and/or exchange information (e.g., for reporting,data gathering, analyzing, and/or troubleshooting purposes) with systemsor devices external to the robotic system 100.

The input-output devices 208 can include user interface devicesconfigured to communicate information to and/or receive information fromhuman operators. For example, the input-output devices 208 can include adisplay 210 and/or other output devices (e.g., a speaker, a hapticscircuit, or a tactile feedback device, etc.) for communicatinginformation to the human operator. Also, the input-output devices 208can include control or receiving devices, such as a keyboard, a mouse, atouchscreen, a microphone, a user interface (UI) sensor (e.g., a camerafor receiving motion commands), a wearable input device, etc. In someembodiments, the robotic system 100 can use the input-output devices 208to interact with the human operators in executing an action, a task, anoperation, or a combination thereof.

The robotic system 100 can include physical or structural members (e.g.,robotic manipulator arms) that are connected at joints for motion (e.g.,rotational and/or translational displacements). The structural membersand the joints can form a kinetic chain configured to manipulate anend-effector (e.g., the gripper) configured to execute one or more tasks(e.g., gripping, spinning, welding, etc.) depending on the use/operationof the robotic system 100. The robotic system 100 can include theactuation devices 212 (e.g., motors, actuators, wires, artificialmuscles, electroactive polymers, etc.) configured to drive or manipulate(e.g., displace and/or reorient) the structural members about or at acorresponding joint. In some embodiments, the robotic system 100 caninclude the transport motors 214 configured to transport thecorresponding units/chassis from place to place.

The robotic system 100 can include the sensors 216 configured to obtaininformation used to implement the tasks, such as for manipulating thestructural members and/or for transporting the robotic units. Thesensors 216 can include devices configured to detect or measure one ormore physical properties of the robotic system 100 (e.g., a state, acondition, and/or a location of one or more structural members/jointsthereof) and/or of a surrounding environment. Some examples of thesensors 216 can include accelerometers, gyroscopes, force sensors,strain gauges, tactile sensors, torque sensors, position encoders, etc.

In some embodiments, for example, the sensors 216 can include one ormore imaging devices 222 (e.g., visual and/or infrared cameras, 2Dand/or 3D imaging cameras, distance measuring devices such as lidars orradars, etc.) configured to detect the surrounding environment. Theimaging devices 222 can generate representations of the detectedenvironment, such as digital images and/or point clouds, that may beprocessed via machine/computer vision (e.g., for automatic inspection,robot guidance, or other robotic applications).

In implementing/executing tasks and/or operations, the robotic system100 (via, e.g., the various circuits/devices described above) cancapture and analyze an image of a designated area (e.g., a pickuplocation, such as inside the truck or on the conveyor belt) to processthe target object 112 of FIG. 1 and the start location 114 of FIG. 1thereof. Similarly, the robotic system 100 can capture and analyze animage of another designated area (e.g., a drop location for placingobjects on the conveyor, a location for placing objects inside thecontainer, or a location on the pallet for stacking purposes) to processthe task location 116 of FIG. 1. For example, the imaging devices 222can include one or more cameras configured to generate images of thepickup area and/or one or more cameras configured to generate images ofthe task area (e.g., drop area). Based on the captured images, therobotic system 100 can determine the start location 114, the tasklocation 116, the associated poses, a packing/placement plan, atransfer/packing sequence, and/or other processing results. Accordingly,the robotic system 100 can derive motion plans to perform tasks and/orinteractions between units/tasks to perform operations.

In some embodiments, for example, the sensors 216 can include positionsensors 224 (e.g., position encoders, potentiometers, etc.) configuredto detect positions of structural members (e.g., the robotic arms and/orthe end-effectors) and/or corresponding joints of the robotic system100. The robotic system 100 can use the position sensors 224 to tracklocations and/or orientations of the structural members and/or thejoints during execution of the task.

Example Robotic Units

FIG. 3 is an illustration of example task units associated with therobotic system 100 of FIG. 1 in accordance with one or more embodimentsof the present technology. The robotic system 100 may include and/or beoperably coupled to a set of robotic units configured toimplement/execute one or more tasks. In some embodiments, the roboticunits can include a devanning unit 302, a sorting unit 304, an objecttransport unit 305, a grouping unit 306, a group transport unit 307, aremoving unit 308, a package opening unit 310, a rack transport unit312, a shelving unit 313, a picking unit 314, a packing unit 316, or acombination thereof.

The devanning unit 302 can be a robotic unit configured to perform orexecute a devanning task 322 by removing target objects from a carrier(e.g., a truck, an airplane, a ship, etc.). In some embodiments, thedevanning unit 302 can include a package-level or a pallet-level roboticarm and/or a lift for lifting the target objects and/or their containers(e.g., pallets and/or other shipping containers). The devanning unit 302can also include a transport system, such as wheels, tracks, rails,etc., configured to move the robotic arm and/or the lift relative to thecarrier.

The sorting unit 304 can be a robotic unit configured to perform asorting task 324 by placing or sending each of the incoming objects todesignated locations/tasks associated with the object and/or accordingto a sequence. In some embodiments, the sorting unit 304 can include atransfer mechanism (e.g., a conveyor) that moves the devanned targetobjects along a path, such as from the devanning unit 302 andthrough/across a manipulation mechanism. The manipulation mechanism caninclude robotic units and/or sensors configured to recognize andmanipulates individual objects on the path according to the recognitionresults. For example, the manipulation mechanism (e.g., a package-levelrobotic arm) can transfer and place the objects at different locationson or outside of the conveyor to form targeted groupings of objectsand/or targeted sequence of objects. Also, the manipulation mechanismcan transfer the objects from the path to one of the object transportunits 305 associated with or assigned to the recognized object.

The object transport unit 305 can be a robotic unit operably coupled tothe sorting unit 304 and configured to transfer objects betweenstations/tasks. For the example illustrated in FIG. 3, the objecttransport unit 305 can transfer the sequenced/grouped objects resultingfrom the sorting task 324 to be further processed for other tasks andthe related units/stations (e.g., locations or areas associated with thetasks and the related units) described below. The object transport unit305 may include a conveyor, a track, and/or a set of locomotive transferunits.

The grouping unit 306 can be a robotic unit configured to perform astorage grouping task 326 by grouping at least a subset of the objects,such as according to categories, types, orders, and/or shippingmanifest, to form grouped sets of the objects. For example, the roboticsystem 100 can control the grouping unit 306 to palletize the incomingobjects according to brand, manufacturer, identifier, size, weight,and/or another category. In other words, a warehouse may receive manydifferent types of packages. Also, the shipped/received groupings ofpackages may have quantities or packing configurations that deviate fromtargeted storage quantities or configurations. Accordingly, the roboticsystem 100 can redistribute the received packages into new groupingsthat match the targeted storage groupings, quantities, and/or packingconfigurations. Each resulting groupings may include correspondingobjects placed on or in containers (e.g., pallets or bins). Accordingly,for storage purposes, the containers may be categorized according to theassociated object groupings. In some embodiments, the robotic system 100may categorize the containers as having a single homogenous grouping ofobjects (e.g., same brand, same identifier, etc.) and/or having multipleor mixed groupings of objects.

In some embodiments, the storage grouping task 326 can include agrouping of two or more subtasks. The subtasks may include (1)transferring or loading empty grouping mechanisms (e.g., pallets orbins) to designated areas, (2) transferring the incoming objects fromthe object transport unit 305 to the grouping areas/mechanisms (e.g.,pallets or bins) that correspond to the types or the instances of theobjects, and/or (3) transferring the loaded grouping mechanism to adesignated location. Accordingly, the grouping unit 306 can include apalletizing robot, such as a package-level robotic arm configured tomanipulate boxes or packages. The palletizing robot may grip and liftthe objects on the object transport unit 305 and place/stack them onpallets located at designated areas. Also, the group transport unit 307can be a robotic unit configured to move the grouped objects, such asbetween palletizing locations and other processing locations (e.g.,depalletizing locations and/or storage locations). For example, thegroup transport unit 307 can include a locomotive robotic unit, such asan Automated Guided Vehicle (AGV), that is configured to pick up andtransport the grouping mechanisms and/or the objects thereon.

The removing unit 308 can be a robotic unit configured to perform agroup manipulation task 328 by rearranging groupings of objects, such asfor adjusting storage groupings and/or for forming outbound objectgroupings. The group manipulation task 328 can be for accessing theobject groupings from the initial storage location and placing them attask stations. For example, the removing unit 308 can include adepalletizing unit, such as a package-level robotic arm configured tomanipulate boxes or packages and remove them from an initial grouping(by, e.g., removing them from a first pallet) and placing them at one ormore different locations (e.g., a second pallet or another conveyor) forstorage.

As an illustrative example of the group manipulation task 328, the grouptransport unit 307 (e.g., the AGV) can bring the pallet and the objectsstored thereon from the storage location to a depalletizing location.The depalletizing unit can transfer the objects from the pallet toanother location for restorage or other processing as described below.

In some embodiments, the devanning task 322, the sorting task 324,and/or the storage grouping task 328 can be sequenced to form areceiving operation 320. The receiving operation 320 can be forreceiving objects from an external provider or source for subsequentprocessing (e.g., grouping and/or storage). For example, the receivingoperation 320 can be for receiving, unloading, and/or storing incomingobjects, such as from manufacturers, warehouses, shipping hubs,distributors, etc.

In one or more embodiments, the group manipulation task 328 can befurther utilized for different operations. For example, the roboticsystem 100 can implement a stocking operation 330 that includes thegroup manipulation task 326. The stocking operation 330 can include asequence of tasks for manipulating, storing, and/or accessing contentsof objects to relocate a task target for further storage or forsubsequent tasks. In other words, the receiving operation 320 canmanipulate the boxes and/or packages for storage and access, and thestocking operation 330 can manipulate the contents within the boxesand/or packages for storage and access.

The stocking operation 330 may also include other tasks, such as apackage opening task 332 and/or a racking task 334. The package openingunit 310 (e.g., a robotic unit) can be configured to perform the packageopening task 332 by opening the container, such as boxes or packagingmaterial, forming or surrounding the object. In some embodiments, thepackage opening unit 310 can be configured to remove or cut packagefasteners (e.g., tape, binding, etc.) and/or open coverings (e.g., boxflaps, plastic wrappings, lids, etc.). In other embodiments, the packageopening unit 310 can be configured to remove a top portion of thepackage, such as by cutting and removing a top portion/surface of thepackage to form an open-top bin and expose items therein. Similarly, therack transport unit 312 (e.g., a robotic unit, such as an AGV) and/orthe shelving unit 313 (e.g., a package-level robotic arm and/or aspecialized AGV) can be configured to perform the racking task 334. Theracking task 334 can be for placing the objects/bins on a storage rackand/or for accessing and removing objects/bins from the storage rack.The racking task 334 can include a rack picking task for removing theobject/bins and transporting them to a different location. The racktransport unit 312 can be configured to transport storage racks betweenstorage locations and loading/unloading locations. The shelving unit 313can be configured to place the objects (e.g., the received objectsand/or the opened objects) on the storage racks and/or remove theobjects from the storage racks.

Similar to the group manipulation task 328, the racking task 334 can befurther utilized for different operations. For example, the roboticsystem 100 can implement a shipping operation 340 that includes theracking task 334. The shipping operation 340 can include a sequence oftasks for grouping objects and/or individual items initially in storageor a different location for outbound transport or shipment. In otherwords, the receiving operation 320 can manipulate the boxes, packages,and/or content items and group them according to orders or shippingmanifest. The grouped objects/items can be subsequently loaded to atransport vehicle and/or shipped to a remote location/facility separatefrom the storage locations.

The shipping operation 340 may also include other tasks, such as apicking task 342, a packing task 344, and/or an outbound grouping task346. A set of robotic units including the removing unit 308, theshelving unit 313, and/or the picking unit 314 (e.g., an item-levelrobotic arm) may be configured to perform the picking task 342 byaccessing and manipulating content items stored/received within objects,such as boxes or packages. For example, the shelving unit 313 and/or theremoving unit 308 may be configured to perform a sub-task by placingstorage containers (e.g., the opened boxes) at processing locations. Thepicking unit 314 (e.g., a robotic arm with a picking end-effector) maygrip and transfer the content items from the storage containers tooutbound containers (e.g., other boxes or packages), such as accordingto orders and/or shipping manifest.

The packing unit 316 may be configured to perform the packing task 344by enclosing the content items and/or the objects for outbound transfer.For example, the packing unit 316 can include a robotic unit configuredto close flaps or lids of the outbound containers, fasten the flaps/lids(via, e.g., tape, fastener, and/or adhesive), wrap the individualoutbound containers, or a combination thereof.

The packing unit 316 (e.g., a package-level robotic arm) may beconfigured to perform the grouping task 346 by placing thepacked/enclosed outbound containers at designated locations. Forexample, the packing unit 316 can load a pallet with a group of outboundcontainers intended for the same vehicle and/or destination location. Insome embodiments, the grouping task 346 may include an additionalsub-task to fasten the grouped containers, such as by wrapping the setof objects with a plastic wrap. A robotic unit (not shown) similar tothe packing unit 316 and/or the AGV may be configured to apply theplastic wrap to the stacked/palleted outbound containers.

For illustrative purposes, the operations have been described withexample task sequences shown in FIG. 3. However, it is understood thatthe operations and/or the tasks may be different. For example, thereceiving operation 320 can include the package opening task 332 and/orthe racking task 334. Additionally or alternatively, the receivingoperation 320 may further exclude the sub-tasks performed by the grouptransport unit 307, and instead, the objects may be placed on the objecttransport unit 305 for further processing. Accordingly, the receivingoperation 320 may transition from package-level manipulations toitem-level operations and store open containers on racks.

Also, as an illustrative example, the shipping operation 340 can includethe package opening task 332 after the racking task 334. In other words,the incoming objects may be stored without opening the objects, such asdescribed above for the package-level receiving operation 320. Theindividual content items may be manipulated and packed as part of theshipping operation 340. Accordingly, the robotic system 100 may bringthe stored packages to picking areas by implementing the racking task334 and open the packages by executing the package opening task 332before the picking task 342.

As a further illustrative example, the shipping operation 340 mayinclude package-level processing. In other words, the incoming objectsmay be stored without opening the objects as described above. The storedobjects may be regrouped onto outbound pallets according to order,vehicle, and/or destination location without the item-levelmanipulations. Accordingly, the package-level outbound grouping task mayinclude the group manipulation task 328 followed by the outboundgrouping task 346.

Example Task/Operation Organization

FIG. 4 is an illustration of an example control diagram for the roboticsystem 100 of FIG. 1 in accordance with one or more embodiments of thepresent technology. The control diagram can illustrate an overallarchitecture for the robotic system 100 and/or the correspondingcomponents. In some embodiments, for example, the robotic system 100 canbe implemented via a management system 402, a storage access system 404,a master controller 408, one or more robotic units, and/or other controlsystems. In other words, the robotic system 100 may be implemented basedon operating the one or more processors 202 of FIG. 2 included in themanagement system 402, the storage access system 404, the mastercontroller 408, the one or more robotic units, and/or the other controlsystems. As described above, the one or more processors 202 may executecomputer-executable instructions stored in the storage devices 204 ofFIG. 2. The storage devices 204 may be included in the management system402, the storage access system 404, the master controller 408, the oneor more robotic units, and/or the other control systems.

In other embodiments, for example, the robotic system 100 can beimplemented via the management system 402 and/or the master controller408 and interface with the storage access system 404, the one or morerobotic units, and/or other control systems. For example, the one ormore processors 202 may execute the computer-executable instructions andcommunicate (e.g., via the communication bus and/or the communicationdevices 206 of FIG. 2) commands, settings, plans, etc. with the storageaccess system 404, the one or more robotic units, and/or other controlsystems to execute the tasks and/or the operations.

The management system 402 can include a set of computing devices (e.g.,the one or more processors 202, the storage devices 204, and/or portionsthereof) configured to manage overall states/conditions of acorresponding location/site. For example, the management system 402 caninclude servers, specialized controllers, desktop computers or portals,and/or other personal or commercial computing devices configured tofunction as a control/management system for a warehouse, a shipping hub,a distribution center, etc. The management system 402 may be located ator in the corresponding location or at a remote location.

The robotic system 100 can control transport objects between taskstations so that the tasks associated with the stations may be performedfor the transported objects. In controlling the transport, for example,the management system 402 and/or the master controller 408 may generatetiming factors (e.g., flags) and/or communicate the timing factors tothe storage access system 404. The storage access system 404 canimplement one or more motion plans or portions thereof to operate thetransport units according to the timing factors from the managementsystem 402 and/or the master controller 408. The storage access system404 can include a set of computing devices (e.g., the one or moreprocessors 202, the storage devices 204, and/or portions thereof)configured to control transport units, such as AGVs 422. For example,the storage access system 404 can include servers, specializedcontrollers, desktop computers or portals, and/or other personal orcommercial computing devices configured to control movements orfunctions of the group transport units 307 of FIG. 3 and/or the racktransport units 312 of FIG. 3.

The master controller 408 can include a set of computing devices (e.g.,the one or more processors 202, the storage devices 204, and/or portionsthereof) configured to control local operations of specific roboticunits and/or the tasks performed by the specific robotic units. Themaster controller 408 can include servers, specialized controllers,desktop computers or portals, and/or other personal or commercialcomputing devices configured to analyze sensor data, determine currentor real-time conditions, and/or derive and implement motion plans forimplementing the tasks.

As an illustrative example, the master controller 408 may receive sensordata representative of objects at the task start location 114 of FIG. 1and identify the incoming object and/or physical attributes thereof(e.g., dimensions, visual appearances, and/or corner/edge locations).The master controller 408 can use the identification results todetermine the task location 116 of FIG. 1 and the corresponding motionplan (e.g., a set of commands and/or settings corresponding to a plannedpath of travel) for transferring the object thereto from the startlocation 114. The master controller 408 can implement the motion planbased on communicating the motion plans or the correspondingcommands/settings to the corresponding robotic units. The robotic unitscan execute the commands/settings to perform the tasks or the sub-tasks.In some embodiments, the master controller 408 may control conveyors 424(e.g., instances of the object transport unit 305 of FIG. 3) and/or thesorting unit 304 of FIG. 3. The master controller 408 may also controlone or more robotic units illustrated in FIG. 3, such as the devanningunit 302, the grouping unit 306, the removing unit 308, the packageopening unit 310, the shelving unit 313, the picking unit 314, and/orthe packing unit 316.

The management system 402, the storage access system 404, and/or themaster controller 408 may be configured to control the correspondingtasks/operations based on operation specifications 406. In someembodiments, the management system 402 may be configured to generate theoperation specifications 406 based on information regarding incomingobjects, currently stored objects, and/or outgoing orders or shippingmanifest. The operation specifications 406 can include details, rules,objectives, timings, and/or interfaces associated with implementationsof the tasks/operations. For example, the operation specification 406can include current quantities and/or storage locations of objectsand/or items within the managed site. Also, the operation specification406 can include grouping or processing locations (e.g., for the sortingunit 304), storage locations and/or storage container/palletidentifications for incoming/received objects and/or reorganizedobjects/items. Further, the operation specification 406 can includeinformation for grouping objects/items for storage and/or outboundshipping.

In some embodiments, the management system 402 may use the operationspecifications 406 to coordinate timings for and/or interactions betweenthe tasks to perform the operations. The management system 402 canderive and/or implement commands, settings, and/or plans for the tasksaccording to the timings and/or the interactions. In other embodiments,the management system 402 may communicate the operation specifications406 to the master controller 408, the storage access system 404, and/orother control devices/systems. The master controller 408, the storageaccess system 404, and/or other control devices/systems can use theoperation specifications 406 to derive and/or implement commands,settings, and/or plans for the tasks.

In some embodiments, the tasks and/or the operations may be performed atdifferent locations within the managed site. Each task and/or operationmay correspond to a production cycle executed at a correspondingstation. For example, the robotic system 100 can control thetasks/operations that correspond to a first production cycle 412, asecond production cycle 414, a third production cycle 416, and/or afourth production cycle 418. In some embodiments, the first productioncycle 412 may correspond to a task, an operation, and/or a portionthereof performed at a palletizing station 432 by one or more associatedrobotic units. Similarly, the second production cycle 414 may correspondto a depalletizing station 434, and the third production cycle 416 maycorrespond to a rack feeding station 436. The fourth production cycle418 may similarly correspond to a rack picking station 438, a piecepicking station 440, and/or a destination station. Details of theproduction cycles and the stations are described below.

FIG. 5A is an illustration of a first example production cycle (e.g.,the first production cycle 412) in accordance with one or moreembodiments of the present technology. Accordingly, FIG. 5A illustratesan example layout and/or function of the palletizing station 432. Insome embodiments, the palletizing station 432 can be configured toperform the storage grouping task 326 of FIG. 3. Accordingly, thepalletizing station 432 may include the grouping unit 306 (e.g., apalletizing unit including a robotic arm with a correspondingend-effector).

The palletizing station 432 may include a source location 502 and one ormore destination locations 504 (e.g., pallet locations). The sourcelocation 502 can include a location where the grouping unit 306 receivesand/or picks up incoming objects. For the example illustrated in FIG.5A, the source location 502 can correspond to an end portion of aningress instance of the conveyor 424 (e.g., an instance of the objecttransport unit 305 of FIG. 3) nearest to the grouping unit 306. Thedestination locations 504 can each be a placement location for agrouping of objects. Object containers, such as bins and/or pallets, maybe placed at the destination locations 504 to receive the objectgroupings. The destination locations 504 and/or the source location 502can be predetermined or spatially fixed relative to the grouping unit306. In some embodiments, the destination locations 504 and/or thesource location 502 can be arranged around (e.g., at least partiallyencircling) and/or within a lateral operating distance associated withthe grouping unit 306.

In some embodiments, the palletizing station 432 may include differenttypes of the destination locations 504, such as single-load locations506 and/or mixed-load locations 508. Each of the single-load locations506 can be designated for loading/grouping a single type of objects. Inother words, the robotic system 100 can place one type of objects on thepallet placed at each of the single-load locations 506. Each of themixed-load locations 508 can be designated for loading/grouping multipledifferent types of objects. In other words, the robotic system 100 canplace multiple types of objects on the pallet placed at each of themixed-load locations 508.

In some embodiments, the mixed-load locations 508 and the single-loadlocations 506 may be predetermined and/or fixed. In other embodiments,the robotic system 100 can dynamically (e.g., during run-time and/oraccording to real-time conditions or processing results) assign a typeto each of the destination locations 504. For example, the roboticsystem 100 (via, e.g., the management system 402 of FIG. 4 and/or themaster controller 408 of FIG. 4) can adjust quantities and/or locationsof the single-load locations 506 and/or the mixed-load locations 508according to real-time conditions. In one or more embodiments, therobotic system 100 can assign identifiers to containers and/or thecorresponding destination locations 504 that specify the assigned type.

The palletizing station 432 can include a prediction queue 510 used todetermine a sequence of the incoming objects. The prediction queue 510can include one or more sensors (e.g., two-dimensional (2D) and/orthree-dimensional (3D) sensors) configured to image one or more objectsas they move towards the source location 502. The prediction queue 510may also include a holding area located before the source location 502that is configured to maintain or house a predetermine number ofobjects. Accordingly, the robotic system 100 can use the predictionqueue 510 to determine identities of predetermined number of objectsthat will sequentially arrive at the source location 502.

As an illustrative example of the first production cycle 412 (e.g., thestorage grouping task 326), the robotic system 100 may obtain a sequenceof the incoming boxes. The robotic system 100 (e.g., the managementsystem 402 and/or the master controller 408) may obtain the sequencefrom a manifest for a received shipment, processing results or statusinformation from the devanning unit 302 of FIG. 3 and/or the sortingunit 304 of FIG. 3. The robotic system 100 may also obtain theinformation by determining the incoming sequence using the predictionqueue 510. The robotic system 100 (e.g., the management system 402, thestorage access system 404, and/or the master controller 408) can assigneach of the incoming objects to one of the destination locations 504according to grouping criteria (e.g., type, brand, objectidentification, etc.).

Continuing with the illustrative example, the master controller 408and/or the management system 402 can request the storage access system404 to assign a container (e.g., a pallet) and a destination locationfor each of the grouping criteria that corresponds to the incomingobjects. Accordingly, the storage access system 404 can provide acontainer identifier (e.g., a pallet identifier) and/or a destinationlocation for an incoming object.

When a container is not currently present at the assigned location, themaster controller 408 and/or the management system 402 can provide amove-in trigger (MoveIn) to the storage access system 404 to bring acontainer to one of the destination locations 504. Based on the move-intrigger, the storage access system 404 can control the AGV 422 of FIG. 4to bring the container to the assigned destination location 504. The AGV422 can provide current location and/or placement status (e.g., taskcompletion status) to the storage access system 404 after placing thecontainer at the assigned location. The storage access system 404 cannotify the master controller 408 and/or the management system 402accordingly.

With the container in place, the master controller 408 and/or themanagement system 402 can control the grouping unit 306 to pick up theobject from the source location 502 and transfer it to the assigneddestination. For example, the master controller 408 and/or themanagement system 402 can derive and/or communicate motion plans and/orcorresponding commands/settings to the grouping unit 306. The groupingunit 306 can execute the received information to grip, lift,horizontally transfer, lower, and release the object to place the objectat the assigned destination. The grouping unit 306 may communicate aplacement status or a task completion status to the master controller408 and/or the management system 402 after transfer of one or moreobjects. The master controller 408 and/or the management system 402 canalso communicate a placement status or a task completion status to thestorage access system 404. The storage access system 404 can track thequantity of objects placed in each container based on the statusupdates.

Once a targeted quantity of objects has been placed on a containerduring the first production cycle 412, the master controller 408 and/orthe management system 402 can provide a move-out trigger (MoveOut) toremove the container from the corresponding destination location. Basedon the move-out trigger, the storage access system 404 can control theAGV 422 to bring the container from the destination location to asubsequent processing location, such as a depalletization station, astorage location, etc. provided by the master controller 408 and/or themanagement system 402. The robotic system 100 can repeat the aboveprocesses until all of the incoming objects have been grouped or nofurther incoming objects are expected.

FIG. 5B is a flow diagram for a method 550 of operating the roboticsystem 100 of FIG. 1 in accordance with one or more embodiments of thepresent technology. The method 550 can be for implementing the firstproduction cycle 412 of FIG. 4 (e.g., the storage grouping task 326 ofFIG. 3). The method 550 can be implemented based on executing theinstructions stored on one or more of the storage devices 204 of FIG. 2with one or more of the processors 202 of FIG. 2. Accordingly, the oneor more processors 202 may implement operations (by, e.g.,generating/sending commands, settings, and/or plans) to control one ormore units (e.g., the grouping unit 306 of FIG. 3, the group transportunit 307 of FIG. 3 such as the AGVs 422 of FIG. 4, the sensors 216 ofFIG. 2, etc.) and/or components therein.

As an illustrative example, the processes illustrated on the left inFIG. 5B may be performed by one or more overseeing devices (e.g., themanagement system 402 and/or the master controller 408) that coordinatethe operations/tasks for a grouping of systems, subsystems, and/ordevices. The processes illustrated on the right in FIG. 5B may beperformed by the storage access system 404. Accordingly, the method 550can illustrate the interactions between the various devices/subsystemsfor the robotic system 100.

At block 552, the one or more overseeing devices can identify incomingobjects. As an illustrative example, the master controller 408 and/orthe management system 402 can identify incoming objects associated witha palletizing task of a corresponding operation. The master controller408 can receive sensor output data (e.g., 2D/3D images) from one or moresensors associated with the prediction queue 510 of FIG. 5A. The mastercontroller 408 and/or the management system 402 can compare the sensoroutput data to the master data 252 of FIG. 2 that includes dimensions,surface images, identifier information, and/or other distinguishablephysical attributes of known/registered objects. The master controller408 and/or the management system 402 can identify or recognize theobjects located in the prediction queue 510 accordingly. In someembodiments, the master controller 408 and/or the management system 402can estimate the identity of the object and/or measure dimensions of theobjects in real-time when the compared aspects of the object are notfound in the master data 252.

At block 554, the one or more overseeing devices can compute a packingsimulation for the incoming objects. For example, the master controller408 and/or the management system 402 can compute the packing simulationsfor grouping the objects according to one or more grouping conditions.The master controller 408 and/or the management system 402 may obtainphysical dimensions (e.g., length, width, and/or height), weight, CoM,and/or other information regarding the identified objects. The mastercontroller 408 and/or the management system 402 can compute the packingsimulation by deriving placement locations within the container and/ormotion plans for placing the object in the container.

To derive the placement locations, the master controller 408 and/or themanagement system 402 can determine targeted grouping goals according toa set of predetermined rules/processes. For example, the mastercontroller 408 and/or the management system 402 may determine testplacement locations as predetermine location on the container (e.g.,peripheral locations and/or center locations). The master controller 408and/or the management system 402 can derive motion plans and/or travelpaths for transferring the object from the source location 502 to thetest placement locations for containers placed at one or more of thedestination locations 504. The master controller 408 and/or themanagement system 402 can evaluate the resulting motion plans, such asaccording to path length, number of maneuvers or direction changes,obstacles, collision likelihoods, and/or other operational criteria. Themaster controller 408 and/or the management system 402 can also evaluatethe stacking/packing arrangements for the containers to determinetargeted capacities that satisfy stability and/or stacking requirements.For example, the master controller 408 and/or the management system 402can simulate various packing/stacking configurations of the objects toachieve an arrangement according to maximum quantity, targetedarrangement, and/or maximum stack height.

At block 556, the one or more overseeing devices can update groupingfactors. The grouping factors can include flags, data, commands, and/orstatus information that represent the container requirements forpacking/storing the incoming objects. Some examples of the groupingfactors may include: flags for beginning preparation of the container(by, e.g., picking up empty or designated/partially-filled containers),an object category associated with the incoming object and/or thecontainer, single/mixed designation for the container, and/or packinglimits/capacity for the container. The master controller 408 and/or themanagement system 402 can communicate the information to the storageaccess system 404. For example, the master controller 408 and/or themanagement system 402 can notify the storage access system 404 toprepare the target storage containers (e.g., pallets) to receive theincoming objects.

At block 582, the storage access system 404 can identify the containersthat will receive the object based on the received grouping factors. Insome embodiments, the storage access system 404 can set a flag thatindicates that preparation activities are being performed by the storageaccess system 404 and/or the AGVs 422. The storage access system 404 candetermine current location of the containers and/or identifiers and/ortypes (e.g., single/mixed) of the containers required for the packingplan. The storage access system 404 may determine thelocations/identifiers by considering the identifiers/types of containersin storage and/or currently at the destination locations 504 of FIG. 5A.

As an illustrative example, the storage access system 404 may determinethe container identifiers as the container already at the single-loadlocation 506 of FIG. 5A when that container corresponds to theidentified incoming object and is available to receive additionalobjects. Also, the storage access system 404 may determine the containeridentifiers as the container already at the mixed-load location 508 ofFIG. 5A when they are designated to receive a mix of objects includingthe currently available object. When multiple corresponding containersare at the destination locations, the storage access system 404 candetermine the container identifier of the container having lowerquantity of objects therein. When no containers (e.g., single and ormixed) at the destination locations 504 correspond to the identifiedincoming object, the storage access system 404 can assign the storedcontainer that includes the smallest quantity of the objects and/or isclosest to the grouping unit 306 to receive the incoming object.

At block 584, the storage access system 404 can prepare containers toreceive the objects. The storage access system 404 can control the AGVs422 to access and transfer the assigned/identified container to one ofthe destination locations 504 or a waiting location for temporarystorage. For example, the storage access system 404 can identify anunoccupied AGV that is nearest to the storage location of the identifiedcontainer. The storage access system 404 can command the identified AGVto pick up the identified container and provide the current storagelocation of the container and a desired target location (e.g., thewaiting area or the one of the destination locations 504). The storageaccess system 404 can track the status of the AGV and update the flag toindicate that the operations/tasks associated preparation activities arecomplete when the AGV arrives at the targeted location.

At block 558, the one or more overseeing devices can track the containerplacement status. For example, the master controller 408 and/or themanagement system 402 can receive container information (e.g., palletidentifiers) representative of the containers prepared by the storageaccess system 404 to receive the incoming objects. When the storageaccess system 404 updates the flag to indicate that operations/tasksassociated with the preparation activities are complete, the mastercontroller 408 and/or the management system 402 can determine a locationfor the container according to the packing simulation. The mastercontroller 408 and/or the management system 402 can communicate thecontainer identifier, the determined destination location, and/or theMoveIn trigger to the storage access system 404 accordingly. Inresponse, as illustrated at block 584, the storage access system 404 cancontrol the AGV to move the container to the determined destinationlocation. The storage access system 404 can update the resulting controlstatus (e.g., location occupancy status, placement status and details ofthe container, and/or other related information for placing thecontainer at the determined location).

At decision block 560, the one or more overseeing devices can determinewhether the container is ready for receiving the object. For example,the master controller 408 and/or the management system 402 can monitorthe control status of the storage access system 404 to determine whetherthe container is ready. The master controller 408 and/or the managementsystem 402 can continue monitoring until the control status indicatesthat the containers have been placed at the designated destinationlocation.

When the containers are ready, as illustrated at block 562, the one ormore overseeing devices can implement object placement. The mastercontroller 408 and/or the management system 402 can derive the motionplan as described above. In some embodiments, the master controller 408and/or the management system 402 can derive or update the placementlocation and the corresponding motion plan in real-time. As anillustrative example, the master controller 408 can receive one or more2D/3D images representative of the container placed at the designateddestination location. The master controller 408 can process the receivedimages, such as by determining height/depth values assigned to a gridsystem or a pixelated model of a placement surface in the container, toderive a placement location of the object. In some embodiments, themaster controller 408 may adjust the motion plan that resulted from thepacking simulation according to the placement location. In otherembodiments, the master controller 408 may derive the object path andthe corresponding motion plan according to the placement location asdescribed above. The master controller 408 can implement the motion planby communicating the motion plan and/or the corresponding commandsand/or settings to the grouping unit 306. The grouping unit 306 canexecute the receive information to transfer an end-effector (e.g.,gripper) to the object, grip the object with the end-effector, lift andlaterally transfer the object, place the object and/or release theobject according to the motion plan.

At block 564, the one or more overseeing devices can update the objectplacement status. For example, the master controller 408 and/or themanagement system 402 can maintain a placement flag that indicateswhether a particular object has been placed in the container. Also, themaster controller 408 and/or the management system 402 can maintain aplacement execution flag that indicates whether the grouping unit 306 isexecuting the motion plan to place an object in the container. Aftereach placement, the master controller 408 and/or the management system402 can determine other information regarding the placed object and/orthe content of the container, such as an identifier for the newly placedobject, a placement location of the newly placed object, an overallshape of the packed set of objects, and/or a quantity of objects in thecontainer.

At block 586, the storage access system 404 can update a containerprofile based on the placement status. The storage access system 404 canmonitor the placement flag and/or the execution flag to identify thatthe object has been placed in the container. When the object has beenplaced in the container, the storage access system 404 can update thecontainer profile that includes details regarding the contents of thecorresponding container. For example, the storage access system 404 canreceive and store the content information from the master controller 408and/or the management system 402 into the container profile. Also, thestorage access system 404 can incrementally increase the object quantitybased on the monitored status(es).

At decision block 566, the robotic system 100 can determine whethersub-tasks or sub-operations associated with the container performed atthe destination location is complete. For example, the storage accesssystem 404, the master controller 408, and/or the management system 402can determine whether the container is full after placement of theobject, such as by comparing the updated object quantity to the quantitylimit determined by the packing simulation and/or a predeterminedstorage packing threshold. Also, the master controller 408 and/or themanagement system 402 can determine whether the container is necessaryor targeted for placement of subsequent incoming objects. When thecontainer is not full and/or targeted for subsequent placements, themaster controller 408 and/or the management system 402 can continueplacing the subsequently incoming objects (e.g., the next object in theprediction queue 510). The master controller 408 and/or the managementsystem 402 can continue with the next placement by repeating theabove-described processes, such as from block 552 and/or block 562.

When the container is full and/or does not correspond to the incomingobjects, as illustrated at block 568, the master controller 408, and/orthe management system 402 can direct the storage access system 404 toremove the container from the destination location, such as by settingthe MoveOut flag. In other words, the robotic system 100 can determinethat no subsequently incoming objects will likely be placed in thecontainer. In response to such determination, the master controller 408and/or the management system 402 can direct the storage access system404 to remove the container from the destination location, such as bysetting the MoveOut flag. In response, as illustrated at block 588, thestorage access system 404 can control the AGV 422 to remove thecontainer from the destination location. The storage access system 404can control the AGV 422 to move to a different destination location, adifferent station, a waiting area, or a storage area according to otheroperational factors or real-time conditions.

In some situations, such as when the removed container was full andother instances of the same type of object remains in the predictionqueue 510, the control flow may proceed to block 582. Accordingly, thestorage access system 404 may subsequently identify another container tobe placed in the newly opened destination location. The method 550 canproceed as described above to place the remaining objects in the updatedcontainer. In some other situations, the control flow can proceed toblock 552 and repeat the above-described processes to place thesubsequently incoming objects in the corresponding containers.

FIG. 6A is an illustration of a second example production cycle (e.g.,the second production cycle 414) in accordance with one or moreembodiments of the present technology. Accordingly, FIG. 6A illustratesan example layout and/or function of the depalletizing station 434. Insome embodiments, the depalletizing station 434 can be configured toperform the group manipulation task 328 of FIG. 3. Accordingly, thedepalletizing station 434 may include the removing unit 308 (e.g., adepalletizing unit including a robotic arm with a correspondingend-effector).

The depalletizing station 434 can be similarly configured as thepalletizing station 432 of FIG. 4 but for removing objects fromcontainers instead of placing objects in the containers. For example,the depalletizing station 434 can include a set of source locations 602where the removing unit 308 receives and/or picks from the containers(e.g., pallets) that include previously packed/stored objects. For theexample illustrated in FIG. 6A, the source locations 602 can correspondto container and/or AGV placement areas. Also, the depalletizing station434 can include one or more destination locations 604 configured totransfer the objects that are removed from (e.g., depalletized) thecontainers to a different location. In some embodiments, each of thedestination locations 604 can include an end portion of an egressinstance of the conveyor 424 (e.g., an instance of the object transportunit 305 of FIG. 3) nearest to the removing unit 308. The sourcelocations 602 and/or the destination location 604 can be predeterminedor spatially fixed relative to the removing unit 308. In someembodiments, the destination location 604 and/or the source locations602 can be arranged around (e.g., at least partially encircling) and/orwithin a lateral operating distance associated with the removing unit308.

The depalletizing station 434 can include a prediction queue 610 used todetermine a set of the incoming containers and/or the correspondingobjects. The prediction queue 610 can include one or more sensors (e.g.,two-dimensional (2D) and/or three-dimensional (3D) sensors) configuredto image the containers/objects as they move towards the sourcelocations 602. The prediction queue 610 may also include a holding arealocated before the source locations 602 that is configured to maintainor house a predetermine number of containers at designated locations.Accordingly, the robotic system 100 can use the prediction queue 610 todetermine identities, quantities, and/or locations of objects that willsequentially arrive at the source locations 602.

As an illustrative example of the second production cycle 414 (e.g., thegroup manipulation task 328), the management system 402 of FIG. 4 and/orthe storage access system 404 of FIG. 4 can determine a trigger forreorganizing containers, such as for combining contents ofpartially-filled containers and/or for filling outgoing shipments.Accordingly, the management system 402 and/or the storage access system404 can identify containers that are subject to the group manipulationtask 328. The storage access system 404 can notify the management system402 and/or the master controller 408 of FIG. 4 of the identifiedcontainers and/or control the AGVs 422 of FIG. 4 to place the identifiedcontainers in the prediction queue 610. The management system 402 and/orthe master controller 408 can receive and/or obtain (via, e.g., sensorsin the prediction queue 610) information regarding the identifiedcontainers and/or the objects therein. The management system 402 and/orthe master controller 408 can derive and implement motion plans fortransferring a targeted instances or quantity of objects from the set ofthe source locations 602 to the destination location 604. The managementsystem 402 and/or the master controller 408 can generate and exchangecoordination signals (e.g., MoveIn, MoveOut, and/or other signals) withthe storage access system 404 for coordinating placement of thecontainers at the source locations 602.

FIG. 6B is a flow diagram for a method 650 of operating the roboticsystem 100 of FIG. 1 in accordance with one or more embodiments of thepresent technology. The method 650 can be for implementing the secondproduction cycle 414 of FIG. 4 (e.g., the group manipulation task 328 ofFIG. 3). The method 650 can be implemented based on executing theinstructions stored on one or more of the storage devices 204 of FIG. 2with one or more of the processors 202 of FIG. 2. Accordingly, the oneor more processors 202 may implement operations (by, e.g.,generating/sending commands, settings, and/or plans) to control one ormore units (e.g., the removing unit 308 of FIG. 3, the group transportunit 307 of FIG. 3 such as the AGVs 422 of FIG. 4, the sensors 216 ofFIG. 2, etc.) and/or components therein.

As an illustrative example, the processes illustrated on the left inFIG. 6B may be performed by one or more overseeing devices (e.g., themanagement system 402 and/or the master controller 408) that coordinatethe operations/tasks for a grouping of systems, subsystems, and/ordevices. The processes illustrated on the right in FIG. 6B may beperformed by the storage access system 404. Accordingly, the method 650can illustrate the interactions between the various devices/subsystemsfor the robotic system 100.

At block 682, the storage access system 404 can identify targetcontainers (e.g., containers in storage and/or at other task locations)for the group manipulation task. As an illustrative example, the one ormore overseeing devices (e.g., the master controller 408 and/or themanagement system 402) can receive an outgoing shipping order andprovide a list of objects or types thereof to the storage access system404. The storage access system 404 can identify containers in storagethat contains the specified objects (e.g., the objects included in theoutgoing shipping order). When multiple containers include the specifiedobjects, the storage access system 404 can select containers closest tothe depalletizing station 434 and/or having a targeted/lowest/highestquantity of objects.

Also, the storage access system 404 can periodically (e.g., according topredetermined timing and/or after task completions) analyze the contentsin the stored containers. The storage access system 404 can initiateobject reorganization when a number of containers havingpartially-filled quantities (e.g., below max threshold capacity of thecontainers) exceed a predetermined reorganization threshold. The storageaccess system 404 can select the partially-filled containers for thegroup manipulation task 328.

At block 684, the storage access system 404 can control placement of thetarget containers at the source locations 602. The storage access system404 can control the group transport unit 307 (e.g., the AGVs 422) tobring the identified containers from their current locations (e.g.,storage locations and/or other task stations) to the depalletizingstation 434. For example, the storage access system 404 can identify anavailable AGV closest to the identified containers. The storage accesssystem 404 can send information (e.g., container identifier/locationand/or the destination for the container) to the identified AGV to pickup and bring the container to the prediction queue 610 of FIG. 6A.

While controlling the placement of the containers, the storage accesssystem 404 can update various statuses and/or information. For example,the storage access system 404 can set status flag(s) that indicatewhether the containers have been placed in the prediction queue 610. Thestorage access system 404 can also provide the container identifier,placement location of the container within the prediction queue 610,type/identifier of the objects in the container, tracked quantity of theobjects in the container, and/or placement positions of the objects inthe container.

At block 652, the one or more overseeing devices can identify incomingobjects. As an illustrative example, the master controller 408 and/orthe management system 402 can receive the container information (e.g.,pallet identifiers) and/or the corresponding incoming objects by thestorage access system 404. The master controller 408 and/or themanagement system 402 can further receive sensor output data (e.g.,2D/3D images) from one or more sensors associated with the predictionqueue 610. The master controller 408 and/or the management system 402can identify or recognize the objects located in the prediction queue610 based on comparing the received sensor data with the master data 252of FIG. 2.

The master controller 408 and/or the management system 402 can interactwith the storage access system 404 for placing the container at one ofthe source locations 602 of FIG. 6A. For example, the master controller408 and/or the management system 402 can identify the object needed at adown-stream station/task. The master controller 408 and/or themanagement system 402 can identify the container in the prediction queue610 that includes the identified object. The master controller 408and/or the management system 402 can further select the source locationfor receiving the identified container. The master controller 408 and/orthe management system 402 can provide to the storage access system 404the identified container, the location within the prediction queue 610for the identified container, and/or the selected source location forthe identified container. In response the storage access system 404 cancontrol the corresponding AGV 422 to move the identified container tothe selected source location. As described above, the storage accesssystem 404 can adjust the flags to reflect whether the containerplacement is ongoing or finished.

At decision block 660, the one or more overseeing devices can determinewhether the container is placed at the selected source location. Forexample, the master controller 408 and/or the management system 402 canmonitor the control status of the storage access system 404 to determinewhether the container is ready. The master controller 408 and/or themanagement system 402 can continue monitoring until the control statusindicates that the containers have been placed at the selected sourcelocation.

When the containers are ready, as illustrated at block 662, the one ormore overseeing devices can implement object removal. The mastercontroller 408 and/or the management system 402 can derive the motionplan for picking up the targeted object from the source location 602 andplace it at the destination location 604 as described above. In someembodiments, the master controller 408 and/or the management system 402can derive or update the placement location and the corresponding motionplan in real-time.

As an illustrative example, the master controller 408 can receive one ormore 2D/3D images representative of the container placed at the selectedsource location. The master controller 408 can process the receivedimages, such as by determining height/depth values assigned to a gridsystem or a pixelated model of a placement surface in the container, toselect an object and/or derive an approach location forapproaching/gripping the object. In some embodiments, the mastercontroller 408 may adjust the motion plan that resulted from the packingsimulation according to the location of the identified/selected objectin the container. In other embodiments, the master controller 408 mayderive the object path and the corresponding motion plan according tothe object location as described above. The master controller 408 canimplement the motion plan by communicating the motion plan and/or thecorresponding commands and/or settings to the removing unit 308. Theremoving unit 308 can execute the receive information to transfer anend-effector (e.g., gripper) to the object, grip the object with theend-effector, lift and laterally transfer the object, place the objectand/or release the object according to the motion plan.

At block 664, the one or more overseeing devices can update the objectplacement status. For example, the master controller 408 and/or themanagement system 402 can maintain a placement flag that indicateswhether a particular object has been placed at the destination location604. Also, the master controller 408 and/or the management system 402can maintain a placement execution flag that indicates whether theremoving unit 308 is executing the motion plan to place an object at thedestination location 604. After each placement, the master controller408 and/or the management system 402 can determine other informationregarding the placed object and/or the content of the source container,such as an identifier for the transferred object, an overall shape ofthe remaining set of objects in the container, and/or a quantity ofremaining objects in the container.

At block 686, the storage access system 404 can update a containerprofile based on the placement status. The storage access system 404 canmonitor the placement flag and/or the execution flag to identify thatthe object has been removed from the container. When the object has beenplaced at the destination location 604, the storage access system 404can update the container profile of the corresponding container byremoving details regarding the removed object. Also, the storage accesssystem 404 can incrementally reduce the object quantity based on themonitored status(es).

At decision block 666, the robotic system 100 can determine whetheroperations/tasks associated with removal of the object(s) from thesource location is complete. For example, the storage access system 404,the master controller 408, and/or the management system 402 candetermine whether the container is empty after placement/removal of theobject. Also, the master controller 408, and/or the management system402 can determine whether the container is necessary or targeted forprocessing of subsequent objects.

When the container is not empty and/or targeted for subsequentprocessing, the master controller 408, and/or the management system 402can continue picking subsequent objects from the containers at thesource location 602. The master controller 408 and/or the managementsystem 402 can continue with the next placement by repeating theabove-described processes, such as from block 662. In some embodiments,the master controller 408, and/or the management system 402 candetermine a desired removal count. The master controller 408 and/or themanagement system 402 can repeat the processes described above for block662 and onward to remove the desired number of objects from the sourcelocation. The master controller 408 and/or the management system 402 candetermine that operations/tasks associated with removal of the object(s)from the source location is complete when the desired number of objectshave been transferred out of the container and to the destinationlocation 604.

When the container is empty and/or does not correspond to thesubsequently targeted objects, the master controller 408, and/or themanagement system 402 can direct the storage access system 404 to removethe container from the source location, such as by setting the MoveOutflag. In other words, the robotic system 100 can determine that nosubsequently targeted objects are available at the source location 602.In response to such determination, the master controller 408, and/or themanagement system 402 can direct the storage access system 404 to removethe container from the source location, such as by setting the MoveOutflag. In response, as illustrated at block 688, the storage accesssystem 404 can generate instructions to control and/or control the AGV422 to remove the container from the source location. The storage accesssystem 404 can generate instructions to control and/or control the AGV422 to move to a different source location, a different station, awaiting area, or a storage area according to other operational factorsor real-time conditions. After removing the container, thereby openingthe source location, the control flow can pass to block 682 and/or 684to bring a new container to the opened source location.

FIG. 7A is an illustration of a third example production cycle (e.g.,the third production cycle 416) in accordance with one or moreembodiments of the present technology. Accordingly, FIG. 7A illustratesan example layout and/or functions of the rack feeding station 436. Insome embodiments, the rack feeding station 436 can be configured toperform the group manipulation task 328 of FIG. 3 and/or the rackingtask 334 of FIG. 3. The rack feeding station 436 may include theshelving unit 313 (e.g., a rack shelving unit including a robotic armwith a corresponding end-effector). In some embodiments, the rackfeeding station 436 can be configured to transfer objects to and placeobject on the storage racks.

The rack feeding station 436 can be similarly configured as thepalletizing station 432 of FIG. 4 but for placing objects and/orcontents therein to storage racks instead of placing objects in thecontainers. For example, the rack feeding station 436 can include a setof source locations 702 where the shelving unit 313 receives the objects(e.g., packages and/or boxes) targeted for transfer to/placement on thestorage racks. For the example illustrated in FIG. 7A, the sourcelocations 702 can correspond to end portions of ingress instances of theconveyors 424 of FIG. 4 (e.g., an instance of the object transport unit305 of FIG. 3) nearest to the shelving unit 313. Also, the rack feedingstation 436 can include one or more destination locations 704 configuredto receive objects that are removed from the source objects. In someembodiments, the destination locations 704 can correspond to placementlocations for racks and/or item containers (e.g., bins and/or objects)thereon. The source locations 702 and/or the destination locations 704can be predetermined or spatially fixed relative to the shelving unit313.

The rack feeding station 436 can include one or more order queues 710.The order queues 710 may precede the source locations 702. In someembodiments, the order queues 710 may include corresponding conveyorsand/or other transport mechanisms that transfer the objects to thesource locations 702. The order queues 710 may each be configured tohold a predetermined number of objects and/or include predeterminedholding locations. Each of the order queues 710 may also include one ormore cameras (e.g., 2D/3D imaging devices) configured toidentify/recognize the objects placed in the order queues 710.

In some embodiments, each of the rack feeding station 436 may beoperably coupled to a cross-station transport unit 712. Thecross-station transport unit 712 (e.g., locomotive robotic units and/orconveyors) can be configured to transport objects across stations. Forthe example illustrated in FIG. 7A, the cross-station transport unit 712may be configured to transport objects from the depalletizing station434 of FIG. 4 (Station B) and/or the piece picking station 440 of FIG. 4(Station E). In other words, the rack feeding station 436 may beconfigured to process objects that were depalletized at thedepalletizing station 434 and/or objects that were filled withdesignated objects at the piece picking station 440. In someembodiments, the rack feeding station 436, via the cross-stationtransport unit 712, may be configured to receive and process openedobjects from the package opening unit 310 of FIG. 3 and/or thecorresponding station for the package opening task 332 of FIG. 3.

The rack feeding station 436 may further include one or more receivingqueues 714 configured to temporarily hold receiving/storage racks (“POD”as illustrated in FIG. 7) before they are placed in the destinationlocations 704 to receive the objects. For example, the robotic system100 can coordinate placement of a sequence of objects in the orderqueues 710. The robotic system 100 can control transport units (e.g.,the AGVs 422 of FIG. 4) prepare/place a sequence of the storage racks inthe receiving queues 714. The receiving queues 714 can correspond to theorder queues 710. Accordingly, the robotic system 100 can increase theefficiency (via, e.g., decreasing time required to place/access thestorage racks) in placing the incoming objects onto the storage racks.

As an illustrative example of the third production cycle 416 (e.g., theracking task 334), the management system 402 of FIG. 4 and/or the mastercontroller 408 of FIG. 4 can identify/recognize the sequence of theincoming objects based on the sensor data from the order queues 710. Themanagement system 402 and/or the master controller 408 can interact withthe storage access system 404 of FIG. 4 for the available storage racksto place the incoming objects and compute the possible sequence of thestorage racks to receive the incoming objects. In some embodiments, themanagement system 402 and/or the master controller 408 can communicatethe sequence of the incoming objects and/or the corresponding racks tothe storage access system 404. The storage access system 404 can prepareand control the transport units (e.g., the AGVs 422) to bring the racksto the receiving queues 714 and the destination locations 704. Themanagement system 402 and/or the master controller 408 can track theprogress/status of the storage access system 404 and coordinate timings(via, e.g., the MoveIn and/or MoveOut flags) for moving the AGVs 422 andthe corresponding racks to/from the receiving queues 714 and thedestination locations 704.

FIG. 7B is a flow diagram for a method 750 of operating the roboticsystem 100 of FIG. 1 in accordance with one or more embodiments of thepresent technology. The method 750 can be for implementing the thirdproduction cycle 416 of FIG. 4 (e.g., the group manipulation task 328 ofFIG. 3 and/or the racking task 334). The method 750 can be implementedbased on executing the instructions stored on one or more of the storagedevices 204 of FIG. 2 with one or more of the processors 202 of FIG. 2.Accordingly, the one or more processors 202 may implement operations(by, e.g., generating/sending commands, settings, and/or plans) tocontrol one or more units (e.g., the shelving unit 313 of FIG. 3, thegroup transport unit 307 of FIG. 3 such as the AGVs 422 of FIG. 4, thesensors 216 of FIG. 2, etc.) and/or components therein.

As an illustrative example, the processes illustrated on the left inFIG. 7B may be performed by one or more overseeing devices (e.g., themanagement system 402 and/or the master controller 408) that coordinatethe operations/tasks for a grouping of systems, subsystems, and/ordevices. The processes illustrated on the right in FIG. 7B may beperformed by the storage access system 404. Accordingly, the method 750can illustrate the interactions between the various devices/subsystemsfor the robotic system 100.

At block 752, the one or more overseeing devices can identify incomingobjects. As an illustrative example, the master controller 408 and/orthe management system 402 can identify the incoming objects (e.g.,depalletized objects or bins resulting from picking tasks) associatedwith the racking task of a corresponding operation. The mastercontroller 408 can receive sensor output data (e.g., 2D/3D images) fromone or more sensors associated with the order queue 710 of FIG. 7Aand/or sensor data from one or more sensors associated with thecross-station transport units 712 of FIG. 7A. The master controller 408and/or the management system 402 can compare the sensor output data tothe master data 252 of FIG. 2 that includes dimensions, surface images,identifier information, and/or other distinguishable physical attributesof known/registered objects. The master controller 408 and/or themanagement system 402 can identify or recognize the objects located inthe order queue 710 accordingly. Also, as an illustrative example, themaster controller 408 and/or the management system 402 can identify theincoming objects based on output status/information from other tasksand/or stations (e.g., the depalletizing station 434 of FIG. 4 and/orthe piece picking station 440 of FIG. 4). Further, the master controller408 and/or the management system 402 can identify the incoming objectsbased on incoming shipping manifests, packing/storage plans, and/oroutgoing orders.

At block 754, the one or more overseeing devices can query for availableracks. In some embodiments, for example, the master controller 408and/or the management system 402 can communicate a predeterminedcommand/message to the storage access system 404 for requesting a listof the available racks. The master controller 408 and/or the managementsystem 402 may also communicate the identified incoming objects to thestorage access system 404 along with and/or instead of the command.

At block 782, the management system 402 can identify available storageracks. In response to the command from the overseeing devices, themanagement system 402 can identify the storage racks that haveopen/available placement location(s) or slot(s) for receiving theincoming objects. For example, the management system 402 can identifythe storage racks that have open/available locations assigned orpredetermined to receive incoming objects. In some embodiments, themanagement system 402 can identify the available storage racks based oncurrent statuses (e.g., filling percentages) of the storage racks. As anillustrative example, when multiple storage racks are assigned toreceive incoming objects, the management system 402 can identify theavailable storage rack as the storage rack with the current statusreflecting the lowest quantity of stored objects and/or correspondingitems. The management system 402 can communicate the identified storageracks and/or other related information (e.g., the current statuses, thecurrent locations, and/or the assigned storage locations of the racks)to the master controller 408 and/or the management system 402.

At block 756, the one or more overseeing devices can compute one or morerack sequences (e.g., a sequential combination of the identified racksor a subset thereof) based on the available storage racks. The mastercontroller 408 and/or the management system 402 can obtain availablerack information representative of the target storage containers (e.g.,storage racks configured to store the bins or the objects having itemstherein) identified by the storage access system as candidates forreceiving the incoming objects for storage. The master controller 408and/or the management system 402 may compute a sequence of the availableracks at the source location(s) 702 of FIG. 7A. For example, the mastercontroller 408 may use a set of predetermined rules/processes to computethe sequence for placing the available racks at the source location(s)702 and the receiving queues 714 of FIG. 7. The master controller 408can compute the rack sequence based on a location and/or a relativesequence of the incoming objects and/or the available racks. The mastercontroller 408 can compute the rack sequence based onreducing/minimizing one or more metrics or factors associated withplacement of the objects.

As an illustrative example, the master controller 408 can compute therack sequence by deriving different test rack sequences and acorresponding placement sequence of the incoming objects. For each testplacement sequence, the master controller 408 may derive motion plansfor placing the incoming objects accordingly. For each motion plan, themaster controller 408 can calculate placement evaluation factors, suchas an object travel distance/time, a number of maneuvers, a type or anumber of input maneuvers, an estimated failure rate, a confidencemeasure, and/or other measures associated with placing the object at thesource locations 702 and/or transferring the object from the sourcelocations 702 to the destination locations 704. The master controller408 may calculate rack sequence measures by combining one or more of theevaluation factors for the motion plans for each of the test placementsequences. The master controller 408 may finalize a set of the racksequences based on the calculated sequence measures. For example, themaster controller 408 can finalize the set of the rack sequences as apredetermined number of sequences with the highest sequence measures.Also, the master controller 408 can finalize the set of rack sequencesas the sequences having the sequence measures above a predeterminedsequence threshold. The one or more overseeing devices can communicatethe finalized set of the rack sequences to the storage access system404.

At block 784, the storage access system 404 can select one or moresequences based on the provided set of sequences. The storage accesssystem 404 can evaluate the finalized set of the rack sequencesaccording to predetermined rules and/or processes. For example, thestorage access system 404 can evaluate the finalized set of the racksequences based on calculating delays, maneuvers, travel distances,and/or other factors associated with accessing and transporting theracks according to the rack sequences. The storage access system 404 canselect one or more of the sequences according to the evaluation of thefinalized set of the rack sequences.

At block 786, the storage access system 404 can prepare the racksaccording to the selected sequence(s). The storage access system 404 mayassign the rack transport unit 312 to the available racks in theselected sequence(s). For example, the storage access system 404 mayassign the AGVs 422 to the racks identified in the selected sequence(s)according to distances between the AGVs 422 and the racks. The storageaccess system 404 can further determine locations, timings, sequences,and/or maneuvers for controlling the AGVs 422 to access the racks andtransport them to the receiving queues 714 and the destination locations704 according to the selected sequence(s). The storage access system 404can control the AGVs 422 accordingly and place the racks in thereceiving queues 714 and the destination locations 704 as identified bythe selected sequence(s). The storage access system 404 can maintain oneor more flags/status information associated with the preparation andcommunicate the one or more flags/status information with the one ormore overseeing devices.

At block 758, the one or more overseeing devices can prepare orderqueues (e.g., the order queues 710) according to the selectedsequence(s) and/or the preparation progress flags/status information.For example, the master controller 408 and/or the management system 402can identify the sequence selected by the storage access system 404. Themaster controller 408 and/or the management system 402 may control thecross-station transport units 712, the order queues 710, and/or one ormore robots at other preceding stations to prepare the order queues. Themaster controller 408 and/or the management system 402 can prepare theorder queues by placing the incoming objects according to a sequencethat matches the selected rack sequence(s). Also, the master controller408 and/or the management system 402 can provide timing informationand/or flags to coordinate placement of the racks. As an illustrativeexample, the master controller 408 can generate or set the MoveIn flagsthat the storage access system 404 can use to control the AGVs 422 andplace the racks in the receiving queues 714 and/or the destinationlocations 704.

Returning to block 786, the storage access system 404 can use theinformation from the one or more overseeing devices to control the racktransport units 312. For example, the storage access system 404 can usethe MoveIn flag as a trigger to control the corresponding AGV 422 andmove the rack from the receiving queue 714 to the destination location704. As described above, the storage access system 404 can update and/ormaintain the status information regarding the rack placement.

At decision block 760, the one or more overseeing devices can determinewhether the storage rack is ready for receiving the object. For example,the master controller 408 and/or the management system 402 can monitorthe control status of the storage access system 404 to determine whetherthe storage rack is ready. The master controller 408 and/or themanagement system 402 can continue monitoring until the control statusindicates that the storage racks have been placed at the designateddestination locations 704.

When the storage racks are ready, as illustrated at block 762, the oneor more overseeing devices can implement object transfer. The mastercontroller 408 and/or the management system 402 can derive the motionplans as described above for picking up the objects from the sourcelocations 702 and placing them at the destination locations 704. In someembodiments, the master controller 408 and/or the management system 402may derive or update the placement location and the corresponding motionplans based on real-time condition of the racks at the destinationlocations 704.

The master controller 408 can implement the motion plan by communicatingthe motion plan and/or the corresponding commands and/or settings to theshelving unit 313. The shelving unit 313 can execute the receiveinformation to transfer an end-effector (e.g., gripper) to the object,grip the object with the end-effector, lift and laterally transfer theobject, place the object and/or release the object according to themotion plan. Accordingly, the master controller 408 can control theshelving unit 313 to place the object on the storage rack.

At block 764, the one or more overseeing devices can update the objectplacement status. For example, the master controller 408 and/or themanagement system 402 can maintain a placement flag that indicateswhether a particular object has been placed on the storage rack. Also,the master controller 408 and/or the management system 402 can maintaina placement execution flag that indicates whether the grouping unit 306is executing the motion plan to place an object in the storage rack.After each placement, the master controller 408 and/or the managementsystem 402 can determine other information regarding the placed objectand/or the content of the storage rack, such as an identifier for thenewly placed object, a placement location of the newly placed object,and/or a quantity of objects on the storage rack.

At block 788, the storage access system 404 can update a rack profilebased on the placement status. The storage access system 404 can monitorthe placement flag and/or the execution flag to identify that the objecthas been placed on the storage rack. When the object has been placed inthe storage rack, the storage access system 404 can update the rackprofile that includes details regarding the contents of thecorresponding storage rack. For example, the storage access system 404can receive and store the content information from the master controller408 and/or the management system 402 into the rack profile. Also, thestorage access system 404 can incrementally increase the object quantitybased on the monitored status(es).

At decision block 766, the robotic system 100 can determine whethersub-tasks associated with the storage rack at the destination locationis complete For example, the storage access system 404, the mastercontroller 408, and/or the management system 402 can determine whetherthe storage rack is full after placement of the object, such as bycomparing the updated object quantity to the predetermined limit for therack. Also, the master controller 408 and/or the management system 402can determine whether the storage rack is necessary or targeted forplacement of subsequent incoming objects. When the storage rack is notfull and/or targeted for subsequent placements, the master controller408 and/or the management system 402 can continue placing thesubsequently incoming objects (e.g., the next object in the order queue710). The master controller 408 and/or the management system 402 cancontinue with the next placement by repeating the above-describedprocesses, such as from block 762.

When the container is full and/or does not correspond to the incomingobjects, as illustrated at block 768, the master controller 408 and/orthe management system 402 can direct the storage access system 404 toremove the storage rack from the destination location, such as bysetting the MoveOut flag. In other words, the robotic system 100 candetermine that no subsequently incoming objects will likely be placed onthe storage rack. In response to such determination, the mastercontroller 408 and/or the management system 402 can direct the storageaccess system 404 to remove the storage rack from the destinationlocation, such as by setting the MoveOut flag. In response, asillustrated at block 790, the storage access system 404 can control theAGV 422 to remove the container from the destination location. Thestorage access system 404 can control the AGV 422 to move to a differentdestination location, a different station, a waiting area, or a storagearea according to other operational factors or real-time conditions.

In some situations, such as when the rack sequence has not beencomplete, the control flow may proceed to block 786. Accordingly, thestorage access system 404 may identify the next racks to be placed atthe destination locations 704 and/or in the receiving queues 714. Themethod 750 can proceed as described above to place the remaining objectsin the updated container. In some other situations, the control flow canproceed to block 752 and repeat the above-described processes to placethe subsequently incoming objects in the corresponding storage racks.

FIG. 8A is an illustration of a fourth example production cycle (e.g.,the fourth production cycle 418) in accordance with one or moreembodiments of the present technology. Accordingly, FIG. 8A illustratesan example layout and/or functions of the rack picking station 438and/or the piece picking station 440. As an illustrative example, thefourth production cycle 418 can be for accessing items that may becontained and stored in multiple different bins (e.g., storage boxes orpackages)/storage racks and for grouping the accessed items into asingle bin. Once the items have been accessed or picked, thecorresponding bins may be transferred to a different station (e.g., therack feeding station 436) where the bins may be placed back on storageracks. The bin including the grouped items can also be transferred to adifferent station for outgoing shipments (e.g., the packing task 344 ofFIG. 3 and/or the outbound grouping task 346 of FIG. 3) and/or forstorage (e.g., the racking task 334 of FIG. 3).

The rack picking station 438 can be configured to perform the groupmanipulation task 328 of FIG. 3 and/or the racking task 334 of FIG. 3.In other words, the rack picking station 438 may be configured to removeobjects or bins from the storage racks. The rack picking station 438 canbe similarly configured as the depalletizing station 434 of FIG. 4 butfor removing bins from storage racks instead of other containers (e.g.,pallets).

The rack picking station 438 may include the shelving unit 313 (e.g., arack shelving unit including a robotic arm with a correspondingend-effector). The shelving unit 313 can access bins from one or morebin source locations 802 and move them to one or more bin destinationlocations 804. For the example illustrated in FIG. 8A, the bin sourcelocations 802 can include placement locations for the targeted binand/or the storage rack having the targeted bin thereon. The bindestination locations 804 can correspond to end portions ofcross-station transport units 808 (e.g., an instance of the objecttransport unit 305 of FIG. 3) nearest to the shelving unit 313. Forexample, the bin destination locations 804 can include end portions ofegressing instances of the conveyors 424 of FIG. 4 configured to carrybins from the rack picking station 438 to another station. The binsource locations 802 and/or the bin destination locations 804 can bepredetermined or spatially fixed relative to the shelving unit 313.

The rack picking station 438 may include one or more rack queues 806.The rack queues 806 may include holding areas for the racks, and therack queues 806 may be located before the bin source locations 802. Forexample, the rack queues 806 can include temporary rack storage areaslocated between the bin source locations 802 and the rack storage area.In some embodiments, the rack queues 806 can include one or more sensors(e.g., the imaging sensors, such as 2D/3D cameras, and/or scanners)configured to identify the rack and/or the bins on the racks. Therobotic system 100 can use the rack queues 806 to sequence the racksand/or buffer the rack storage to improve efficiencies for the groupmanipulation task 328, such as by reducing access times associated withthe shelving unit 313 accessing the targeted bins.

The piece picking station 440 can be configured to perform the pickingtask 342 of FIG. 3 by picking/removing items from within the bins andtransferring the picked items to destination locations or destinationbins. The piece picking station 440 can be similarly configured as thepalletizing station 432 of FIG. 4 but for removing and transferringitems contained within objects/bins instead of transferring theobjects/bins themselves.

The piece picking station 440 may include the picking unit 314 (e.g., apiece-picking unit including a robotic arm with a correspondingend-effector) for performing the picking task 342. The picking unit 314can access targeted items from bins that are placed at one or more itemsource locations 812 and move them to output bins located at one or moreitem destination locations 814. For the example illustrated in FIG. 8A,the item source locations 812 can include end portions of thecross-station transport units 808 opposite the bin destination locations804. In other words, the cross-station transport units 808 can transportthe bins accessed by the shelving unit 313 to the piece picking station440, and the picking unit 314 can pick the items from the bins that areon the cross-station transport units 808. The destination locations 814can be locations for item-receiving bins designated to receive thepicked items. Some examples of the destination locations 814 can includedesignated locations on the floor and/or designated locations on otherinstances of the object transport unit 305. The item source locations812 and/or the item destination locations 814 can be predetermined orspatially fixed relative to the picking unit 314.

The piece picking station 440 may include other instances of the objecttransport unit 305 configured to transport the accessed bins to otherstations for subsequent processing. As an illustrative example, thepiece picking station 440 can include the cross-station transport units712 configured to transport the bins to the rack feeding station 436 ofFIG. 4. Once the picking unit 314 completes the picking task 342 for thebin, the cross-station transport units 712 can transport the bin to therack feeding station 436. The transported bin can be placed on a storagerack at the rack feeding station 436 as described above.

Similarly, the item-receiving bins can be transported via otherinstances of the object transport unit 305 to other stations forsubsequent processing. For example, the item-receiving bins can betransported from the item destination locations 814 to the rack feedingstation 436 for placement on storage racks, such as for concluding thestocking operation 330 of FIG. 3 (e.g., to reorganize or redistributethe items). Also, the item-receiving bins can be transported for packingtask 344 of FIG. 3 and/or outbound grouping task 346 of FIG. 3, such asfor concluding the shipping operation 340 of FIG. 3.

In some embodiments, as illustrated in FIG. 8A, one control device(e.g., one instance of the master controller 408) may control both theshelving unit 313 and the picking unit 314 (e.g., two robotic arms) andthe corresponding transport units for the fourth production cycle 418.The storage access system 404 may have access (via, e.g., the managementsystem 402) to picking objectives (e.g., outgoing orders) targeted forthe picking unit 314. In some embodiments, the storage access system 404and/or the master controller 408 may produce a production queue (via,e.g., the rack queues 806) and/or an order queue (via, e.g., the pickingqueues 816). Based on the order queue, the master controller 408 canderive schedules/sequences for the racks and share the results with thestorage access system 404. The storage access system 404 can selectand/or finalize the rack schedule/sequence and control placement of theracks at the rack queues 806 and/or the bin source locations 802accordingly. The master controller 408 may provide triggers (e.g.,MoveIn and/or MoveOut) to the storage access system 404 for controllingthe placement of the racks. The master controller 408 can control theshelving unit 313 based on the placement of the racks, and then controlthe picking unit 314 based on the tasks performed by the shelving unit313 and/or the cross-station transport units 808. The master controller408 can update the storage access system 404 with the number of itemsremoved from the accessed bin. Accordingly, the storage access system404 can update profile/content information for the accessed bin.

FIG. 8B is a flow diagram for a method 850 of operating the roboticsystem of FIG. 1 in accordance with one or more embodiments of thepresent technology. The method 850 can be for implementing the fourthproduction cycle 418 of FIG. 4 (e.g., the picking task 342 of FIG. 3and/or the group manipulation task 328 of FIG. 3). The method 850 can beimplemented based on executing the instructions stored on one or more ofthe storage devices 204 of FIG. 2 with one or more of the processors 202of FIG. 2. Accordingly, the one or more processors 202 may implementoperations (by, e.g., generating/sending commands, settings, and/orplans) to control one or more units (e.g., the shelving unit 313 of FIG.3, the picking unit 314 of FIG. 3, transport units, the sensors 216 ofFIG. 2, etc.) and/or components therein.

As an illustrative example, the processes illustrated on the left inFIG. 8B may be performed by one or more overseeing devices (e.g., themanagement system 402 and/or the master controller 408) that coordinatethe operations/tasks for a grouping of systems, subsystems, and/ordevices. The processes illustrated on the right in FIG. 8B may beperformed by the storage access system 404. Accordingly, the method 850can illustrate the interactions between the various devices/subsystemsfor the robotic system 100.

At block 882, the storage access system 404 can identify piece ordersintended for fulfilment by the fourth production cycle 418. For example,the storage access system 404 can receive an outgoing/customer order, areorganization plan, or another item grouping plan from the managementsystem 402. The storage access system 404 can identify details regardingthe received orders, such as item identifiers, item types or categories,item quantities, grouped items, and/or grouping sequences for eachcontainer and/or a sequence of containers.

At block 884, the storage access system 404 can generate queue and/orstorage data associated with the identified items. For example, thestorage access system 404 can identify storage locations and/or currentlocations of bins having the identified items therein. In identifyingthe locations, the storage access system 404 can use the identifieddetails to search maintained profiles for bins/containers/objects/racksand the contents therein. The storage access system 404 can identify thecorresponding storage units that include the identified items and theirtracked and/or designated locations.

In some embodiments, the storage access system 404 can communicate theidentified storage units to the one or more overseeing devices. The oneor more overseeing devices may use the identified storage units and/ortheir locations to generate sequences for queues (e.g., the rack queues806 of FIG. 8A and/or the picking queues 816 of FIG. 8A). In otherembodiments, the storage access system 404 can generate the queuesequences and communicate the generated information to the one or moreoverseeing devices. The robotic system 100 can generate the queuesequences each including an ordered combination of the storage unitsand/or corresponding placement timings.

The robotic system 100 may generate one or more queue sequences for therack queues 806 as the ordered combination of the racks having binsthereon that include the targeted items. The robotic system 100 mayfurther generate one or more queue sequences for the picking queuesbased on the queue sequences for the rack queues 806.

As an illustrative example, the robotic system 100 can generate thequeue sequences based on the current/storage locations for thecorresponding storage units and according to predeterminedrules/processes. The robotic system 100 can derive test sequences forplacing the storage units at the rack queues 806 and/or the bindestination locations 804. Based on the test sequences, the roboticsystem 100 can derive associated sequences for the picking queues 816 asan ordered combination of the bins transferred from the placed storageracks. The robotic system 100 can further derive the correspondingrobotic unit actions, robotic unit maneuvers, travel paths, traveltimes, and/or other costs associated with placing the storage unitsaccording to the test sequences and/or placing the bins according to theassociated sequences. The robotic system 100 can select/finalize a setof the test sequences as the queue sequences according to the derivedcosts. For example, the finalized queue sequences can be a predeterminednumber of the test sequences having the lowest costs and/or the testsequences having costs below a predetermined placement threshold.

At block 852, the one or more overseeing devices can identify the queuesequence and/or storage information. As described above, the mastercontroller 408 and/or the management system 402 can generate theinformation or receive the information from the storage access system404. The queue sequence can include rack queue informationrepresentative of a sequence of storage racks having bins thereon thatinclude ordered items. The queue sequence can also include picking queueinformation representative of a sequence of the bins.

The master controller 408, the management system 402, and/or the storageaccess system 404 can interact with each other to implement the rackand/or placement according to the queue sequence. At block 854, themaster controller 408, the management system 402, and/or the storageaccess system 404 can transfer and place the bin for piece picking. Themaster controller 408, the management system 402, and/or the storageaccess system 404 can transfer and place the bin (e.g., a de-rackingtask) based on one or more processes similar to processes describedabove for the method 650 of FIG. 6B. For example, the storage accesssystem 404 can place the targeted racks in the rack queues 806 and/orthe bin source locations 802 according to timing control provided by themaster controller 408 and/or the management system 402. Based on theplacement status of the racks, the master controller 408 and/or themanagement system 402 can control (via, e.g., derived motion plans) theshelving unit 313 to transfer the targeted bins from the racks at thebin source locations 802 to the bin destination locations 804. Forexample, the master controller 408 may operate the shelving unit 313 toremove the target bin from the storage rack and place the bin on thedestination location according to the picking queue information.

At block 856, the one or more overseeing devices can coordinate bintransfer for picking tasks. For example, the master controller 408and/or the management system 402 can track movements of the shelvingunit 313 to determine when the bins are placed at the bin destinationlocations 804, such as the cross-station transport units 808.Accordingly, the master controller 408 and/or the management system 402can control the cross-station transport units 808 to move the placedbins from the rack picking station 438 (e.g., the bin destinationlocations 804 therein) to the piece picking station 440 (e.g., the itemsource locations 812 therein).

At decision block 860, the one or more overseeing devices can determinewhether the bin is ready for the piece picking task. For example, themaster controller 408 and/or the management system 402 can track themovements of the cross-station transport units 808 and/or the binsthereon. The master controller 408 and/or the management system 402 cancontinue transferring the target bins to the piece picking station 440and/or the item source locations 812 therein.

When the targeted bins are ready, such as illustrated at block 862, theone or more overseeing devices can implement item transfers. Forexample, the master controller 408 and/or the management system 402 canimplement a transport plan for operating the object transport unit 305to transport the placed bin from the rack picking station 438 to theitem source location of the piece picking station 440. The mastercontroller 408 and/or the management system 402 may obtain, via 2D/3Dsensors at the item source locations 812, current conditions (e.g., itemposes) within the bins at the item source locations 812. The mastercontroller 408 and/or the management system 402 can recognize the itemsbased on the sensor data and derive motion plans and/or the travel pathsfor transferring the items from the bin at the item source locations 812to the bins at the item destination locations 814. The motion plans cancorrespond to commands, settings, and/or sequences thereof for grippingthe items, lifting and laterally transferring the items, and/or loweringand placing the items at the item destination locations 814. The mastercontroller 408 and/or the management system 402 can use the motion plansto control the picking unit 314 to transfer the items.

At block 864, the one or more overseeing devices can update theplacement status based on tracking the progress of the motion plan. Forexample, the master controller 408 and/or the management system 402 caninteract with the picking unit 314 to track the progress of the motionplan. The master controller 408 and/or the management system 402 canupdate the placement status when the item is placed in the bin at thecorresponding item destination location 814. The master controller 408and/or the management system 402 can further interact with the storageaccess system 404 according to the placement status. At block 888, thestorage access system 404 can update profiles for the bins at the itemsource locations 812 and/or the bins at the item destination locations814. For example, the storage access system 404 can update the profilesby reducing the item count and/or updating the item locations for thebins at the item source locations 812. Also, the storage access system404 can update the profiles by increasing the item count and/or updatingthe item locations for the bins at the item destination locations 814.

At decision block 866, the one or more overseeing devices can determinewhether operations/tasks associated with the bins at the item sourcelocations 812 is complete. In other words, the master controller 408and/or the management system 402 can determine whether all planned itemshave been removed from the sourcing bin. In some embodiments, the mastercontroller 408 can determine a number of items designated to be pickedfrom each incoming bin. The master controller 408 can track theplacement status to determine the number of items removed from the binat the item source locations 812. When the number of removed items isless than the targeted number of items, the master controller 408 cancontinue to implement the item transfer as illustrated by the feedbackloop to block 862.

When the targeted tasks for the bin is complete, such as illustrated atblock 868, the one or more overseeing devices can coordinate removal ofthe bins from the item source locations 812. For example, the mastercontroller 408 and/or the management system 402 can control the objecttransport units 305 of FIG. 3 (e.g., the cross-station transport units712 of FIG. 7A and/or the cross-station transport units 808) to removethe bins and/or transfer the bins to the next station (e.g., the rackfeeding station 436 of FIG. 7A to return the bin to the storage rackand/or the storage location).

Once the bin has been removed and/or along with removal of the bin, theone or more overseeing devices can transfer the subsequent bin for thenext picking task. As illustrated by the feedback loop to block 856, themaster controller 408 and/or the management system 402 can transfer thesubsequent bin for picking along with coordinating the removal of thefinished bin. For example, the master controller 408 and/or themanagement system 402 can operate the cross-station transport units 808to transfer the subsequent bin to the item source locations 812 whileremoving the existing bin. The master controller 408 and/or themanagement system 402 can use the status of the placements/removals tooperate the shelving unit 313 and/or the corresponding tasks. The mastercontroller 408 and/or the management system 402 can further communicatethe status of the bins with the storage access system 404. Accordingly,the storage access system 404 can coordinate rack removals/placementswith respect to the rack queues 806 and/or the bin destination locations804. When the last bin in the queues for the piece order has beentransferred from the rack, the flow can return to block 882 to processsubsequent orders.

Example Transitions Between Tasks

FIGS. 9A and 9B are illustrations of example task transitions inaccordance with one or more embodiments of the present technology. FIG.9A illustrates an example layout 900 for the environment in which therobotic system 100 of FIG. 1 may operate. The environment may includeone or more storage areas 902, one or more transition queues 904, and/orone or more task stations 906. Each storage area 902 can be configuredto store targeted objects/items. In some embodiments, the storage area902 can be configured to store group storage units 912 (e.g., storageracks and/or pallets) that hold multiple individual storage units 914(e.g., the objects, such as packages, boxes, bins, and/or othercontainers) and/or items therein. For example, the robotic system 100can control/implement the receiving operation 320 of FIG. 3 and/or thestocking operation 330 of FIG. 3 to place the targeted objects/items atthe storage area 902.

The transition queues 904 can include temporary holding areas configuredto provide access to targeted racks, objects/bins, and/or itemsaccording to a determined sequence. The transition queues 904 can beused as access buffer between the storage area 902 and the task stations906. Some examples of the transition queues 904 can include pickingqueues 816 of FIG. 8A, the rack queues 806 of FIG. 8A, the order queues710 of FIG. 7A, and/or other queues described above. Each of the taskstations 906 can include an area configured to perform a task, asub-task, an operation, or a combination thereof. Some examples of thetask stations 906 can include the palletizing station 432 of FIG. 4, thedepalletizing station 434 of FIG. 4, the rack feeding station 436 ofFIG. 4, the rack picking station 438 of FIG. 4, the piece pickingstation 440 of FIG. 4, the destination station 442 of FIG. 4, and/orother stations for tasks/operations described above.

As illustrated in FIG. 9A, the task stations 906 and/or the transitionqueues 904 can correspond to the rack picking station 438, the piecepicking station 440, and/or the rack feeding station 436. The taskstations 906 and/or the transition queues 904 can correspond to analternative embodiment of the rack picking station 438 and/or the piecepicking station 440. For example, the robotic system 100 can controldifferent types of AGVs to transport the racks between the storage area902 and the transition queues 904, and transport objects between theracks at the transition queues 904 and the task stations 906. The AGVscan follow storage access paths 922 between the storage area 902 and thetransition queues 904, and/or follow task access paths 924 between thetransition queues 904 and the task stations 906. The robotic system 100(e.g., the AGVs 422 and/or the storage access system 404) can determineor access predetermined locations of the access paths 922 and/or theaccess paths 924.

FIG. 9B illustrates an example of the task stations 906. The illustratedexample can be an alternative embodiment of the piece picking station440, the rack feeding station 436, and/or a portion thereof. The taskstation 906 can include a transfer unit 934, such as the grouping unit306 of FIG. 3, the picking unit 314 of FIG. 3, the removing unit 308,and/or other robotic units described above configured to perform thecorresponding tasks.

The task stations 906 can include one or more access locations 930 thatrepresent predetermined stopping locations for the AGVs 422. Forexample, the access locations 930 can be the end locations of the taskaccess paths 924. Accordingly, by placing the AGVs 422 at the accesslocations 930, the robotic system 100 can place target containers 932(e.g., pallets and/or bins) at predetermined task locations (e.g.,source/destination locations described above). The robotic system 100can operate the transfer unit 934 to access or place the targetcontainers 932 to/from the access locations 930. The transfer unit 934can perform the tasks according to corresponding task locations 936.

As an illustrative example of the picking task 342, the AGVs 422 canbring the targeted bin to the access locations 930. The picking unit 314of FIG. 3 (e.g., an instance of the transfer unit 934) can pick itemsfrom the targeted bin and transfer them to the corresponding tasklocations 936 (e.g., the target bin at the item destination locations814 of FIG. 8A). As an illustrative example of the storage grouping task326 of FIG. 3, the corresponding task locations 936 can be the objectpick up location (e.g., the source location 502 of FIG. 5A) and theaccess locations 930 can be the object placement location (e.g., thedestination locations 504 of FIG. 5A). Alternatively, the correspondingtask locations 936 can be the object placement location (e.g., thedestination location 604 of FIG. 6A) and the access locations 930 can bethe object pick up location (e.g., the source location 602 of FIG. 6A)for a portion of the depalletizing task.

FIG. 9C is an illustration of example transport units in accordance withone or more embodiments of the present technology. In some embodiments,the AGVs 422 of FIG. 4 can include a rack transport unit 942 and/or ashelf access unit 944. The rack transport unit 942 can be configured totransport the storage racks between designated locations, such asbetween the storage area 902 of FIG. 9A and the transition queue 904 ofFIG. 9A. In one or more embodiments, the rack transport unit 942 caninclude a locomotive robot configured to contact and lift the targetedracks for transport.

The shelf access unit 944 can be configured to transport objects or binsto/from racks and other corresponding locations. The shelf access unit944 may include access mechanisms, such as arms and/or fork liftsconfigured to place and/or remove objects/bins from shelves. Forexample, the access mechanism for the shelf access unit 944 can includea height adjustable platform with extendable arms attached thereto. Theheight adjustable platform can be raised or lowered alongvertically-oriented rails of the shelf access unit 944 to acorresponding height of a rack shelf that includes the target object(e.g., bin/container). Once the height adjustable platform is at thecorresponding height, the extendable arms can be extended and securingflaps attached at distal ends of the extendable arms can be engaged(e.g., folded or rotated) to secure the back of the target object (e.g.,the side of the bin/container facing away from the shelf access unit944). The extendable arms and the securing flaps may engage and load thetarget object onto the height adjustable platform when the extendablearms are retracted towards the shelf access unit 944. In someembodiments, the shelf access unit 944 can transport the objects or binsto/from racks in the storage area 902 and/or the transition queue 904.The shelf access unit 944 may transport the objects to/from the taskstations 906. In some embodiments, the robotic system 100 (via, e.g.,the storage access system 404) can control the shelf access unit 944 todirectly remove the one or more bins from the storage racks in thestorage area and transport the removed bins to the picking station.

Example Operation Flow

FIG. 10 is a flow diagram for a method 1000 of operating the roboticsystem 100 of FIG. 1 in accordance with one or more embodiments of thepresent technology. The method 1000 can be for performing andcoordinating operations and the multiple tasks for each of the tasks.The method 1000 can be implemented based on executing the instructionsstored on one or more of the storage devices 204 of FIG. 2 with one ormore of the processors 202 of FIG. 2. Accordingly, the one or moreprocessors 202 may implement operations (by, e.g., generating/sendingcommands, settings, and/or plans) to control one or more units (e.g.,the robotic units, the sensors 216 of FIG. 2, etc.) and/or componentstherein.

At block 1002, the robotic system 100 can identify an operation triggerfor performing an operation. In some embodiments, the management system402 of FIG. 4 and/or the master controller 408 of FIG. 4 may identifythe operation trigger based on one or more external inputs and/oroperator inputs. The available operations (e.g., the receiving operation320, the stocking operation 330, etc. illustrated in FIG. 3) may eachhave one or more predetermined conditions assigned as triggers. Forexample, an arrival of a shipping vehicle can be an operation triggerfor the receiving operation 320. Object/item counts for one or morecontainers (e.g., pallets, bins, storage racks) falling belowmaintenance may trigger the stocking operation 330. Reception of anorder may trigger the shipping operation 340.

At block 1004, the robotic system 100 can determine a target conditionfor the operation. The target condition can represent a goal or anobjective associated with each operation (e.g., an end state ofcompleting the operation). For example, the target condition for thereceiving operation 320 can include groupings of incoming objects and/orstorage locations for the incoming objects. The target condition for thestocking operation 330 can include updated groupings, targeteditem/object counts per container, and/or updated storage locationsalready in storage. The target condition for the shipping operation 340can include groupings of the ordered objects/items. The managementsystem 402, the master controller 408, and/or the storage access system404 of FIG. 4 can determine the target condition according to one ormore predetermined rules/processes.

At block 1006, the robotic system 100 can identify a sequence of tasksand/or corresponding stations for the identified operation. For example,the management system 402 and/or the master controller 408 can identifya predetermined set/sequence of tasks associated with the triggeredoperation. Accordingly, the management system 402 and/or the mastercontroller 408 can identify the robotic units, the subsystems, and/orthe task stations associated with the tasks.

At block 1008, the robotic system 100 can perform the tasks according tothe identified sequence. In some embodiments, the management system 402can communicate information for triggering and performing the tasks tothe master controller 408, the storage access system 404, and/or therobotic units. At each task station, the robotic system 100 canimplement the corresponding task.

At block 1010, the robotic system 100 can obtain an access sequence foreach task. For example, the management system 402 and/or the mastercontroller 408 can obtain the access sequence based on computing packingsimulations, tracking container placement status, coordinating incomingobjects, computing rack sequences, preparing order queues, and/oridentifying queue/storage information as described above.

At block 1012, the robotic system 100 can implement access of targetobjects/bins/racks at corresponding start locations at the correspondingtask location. For example, the management system 402 and/or the mastercontroller 408 can implement the access based on generating one or moreaccess coordination factors (via, e.g., activating the MoveIn flag) asdescribed above. The storage access system 404 and/or the mastercontroller 408 can control the object transport units 305 (e.g., theAGVs 422 of FIG. 4 and/or the conveyors 424 of FIG. 4) based on the oneor more access coordination factors to place the targetobjects/bins/racks at the start locations.

At block 1014, the robotic system 100 can control key actions of thetask. For example, the master controller 408 can implement the task,such as by communicating the corresponding motion plan and/orcorresponding commands/settings to the robotic unit in the task station.The master controller 408 may communicate the information to one or moreunits illustrated in FIG. 3, such as the devanning unit 302, the sortingunit, the object transport unit 305, the grouping unit 306, the grouptransport unit 307, the removing unit 308, the package opening unit 310,the rack transport unit 312, the shelving unit, the picking unit 314,and/or the packing unit 316. The robotic units can execute the motionplan or the corresponding commands/settings to perform the tasks.

At block 1016, the robotic system 100 can transfer manipulatedobjects/items to a subsequent task/station. For example, the managementsystem 402 and/or the master controller 408 can implement the transferbased on generating one or more coordination factors (via, e.g.,activating the MoveOut flag) as described above. The storage accesssystem 404 and/or the master controller 408 can control the objecttransport units 305 (e.g., the AGVs 422 and/or the conveyors 424) basedon the one or more access coordination factors to remove the targetobject/bins/items that was manipulated by the key action.

The robotic system 100 can repeat the above-described processes toperform subsequent task(s). The robotic system 100 can transport variousobjects, racks, and/or items between task stations and implement thetasks as described above to perform the triggered operations.

The robotic system 100 can coordinate a sequence of tasks for performingdifferent operations. As described above, the robotic system 100 cancoordinate various actions to sequentially perform the tasks withminimal to no operator inputs. Accordingly, the robotic system 100 canprovide autonomous or near-autonomous management of operations atwarehouses and/or shipping centers.

CONCLUSION

The above Detailed Description of examples of the disclosed technologyis not intended to be exhaustive or to limit the disclosed technology tothe precise form disclosed above. While specific examples for thedisclosed technology are described above for illustrative purposes,various equivalent modifications are possible within the scope of thedisclosed technology, as those skilled in the relevant art willrecognize. For example, while processes or blocks are presented in agiven order, alternative implementations may perform routines havingsteps, or employ systems having blocks, in a different order, and someprocesses or blocks may be deleted, moved, added, subdivided, combined,and/or modified to provide alternative or sub-combinations. Each ofthese processes or blocks may be implemented in a variety of differentways. Also, while processes or blocks are at times shown as beingperformed in series, these processes or blocks may instead be performedor implemented in parallel, or may be performed at different times.Further, any specific numbers noted herein are only examples;alternative implementations may employ differing values or ranges.

These and other changes can be made to the disclosed technology in lightof the above Detailed Description. While the Detailed Descriptiondescribes certain examples of the disclosed technology as well as thebest mode contemplated, the disclosed technology can be practiced inmany ways, no matter how detailed the above description appears in text.Details of the system may vary considerably in its specificimplementation, while still being encompassed by the technologydisclosed herein. As noted above, particular terminology used whendescribing certain features or aspects of the disclosed technologyshould not be taken to imply that the terminology is being redefinedherein to be restricted to any specific characteristics, features, oraspects of the disclosed technology with which that terminology isassociated. Accordingly, the invention is not limited, except as by theappended claims. In general, the terms used in the following claimsshould not be construed to limit the disclosed technology to thespecific examples disclosed in the specification, unless the aboveDetailed Description section explicitly defines such terms.

Although certain aspects of the invention are presented below in certainclaim forms, the applicant contemplates the various aspects of theinvention in any number of claim forms. Accordingly, the applicantreserves the right to pursue additional claims after filing thisapplication to pursue such additional claim forms, in either thisapplication or in a continuing application.

What is claimed is:
 1. A method for operating a robotic system, themethod comprising: identifying a set of two or more availableoperations, each operation including a unique sequence of two or moretasks that include at least a group manipulation task and/or a rackingtask arranged adjacent to each other and/or adjacent to a unique task incomparison to other operations in the set; obtaining a target conditionrepresentative of an end state of completing an operation selected fromthe set of two or more available operations; based on the targetcondition, identifying a sequence of tasks from the set of two or moreavailable operations; obtaining an access sequence based on theidentified sequence of tasks, wherein the access sequence represents anorder for placing target storage containers at a first source locationof a first task station; and implementing the identified sequence oftasks according to the unique sequence for (1) accessing a task targetfrom one of the target storage containers at the first source locationand transferring the task target to a destination location at the firsttask station and (2) accessing the task target from the second sourcelocation and for manipulating the task target in completing theoperation.
 2. The method of claim 1, wherein: the set of two or moreoperation includes a receiving operation for receiving and storing a setof incoming objects in a storage area, and the obtained target conditionis representative of the receiving operation and includes a storagelocation for a set of the incoming objects.
 3. The method of claim 2,wherein the receiving operation includes: a devanning task for removingthe incoming objects from a carrier, a sorting task for placing theincoming objects according to a sequence and/or at subsequent taskstations, a storage grouping task for palletizing at least a subset ofthe incoming objects to form the set of incoming objects, and the groupmanipulation task for placing the palletized set of incoming objects atthe storage location.
 4. The method of claim 1, wherein: the operationincludes a stocking operation for relocating the task target for furtherstorage or subsequent tasks, and the obtained target condition isrepresentative of the stocking operation and includes an updatedlocation for the task target, wherein the updated location is differentfrom an initial storage location.
 5. The method of claim 4, wherein thestocking operation includes: the group manipulation task for accessingthe task target from the initial storage location and placing the tasktarget at a rack feeding station, and the racking task for placing thetask target on a storage rack and for placing the storage rack thatincludes the task target according to the updated location.
 6. Themethod of claim 4, wherein the stocking operation further comprises apackage opening task for exposing items in the task target.
 7. Themethod of claim 1, wherein: the operation includes a shipping operationfor grouping task targets for outbound shipment, and the obtained targetcondition is representative of the shipping operation and includes ashipping order representing a grouping of the task targets.
 8. Themethod of claim 1, wherein the operation includes: a rack picking taskfor accessing the task targets and transporting the task targets to apicking station, wherein accessing the task targets include removing oneor more bins from one or more storage racks in or from the storage area,and a piece picking task for transferring the task targets from thecorresponding one or more bins to an outbound container.
 9. The methodof claim 8, wherein the rack picking task includes sending informationto a storage access system for controlling one or more Automated GuidedVehicles (AGVs) to directly remove the one or more bins from the storagearea and transport the one or more bins to a source location of thepicking station.
 10. The method of claim 8, wherein the operationincludes: the racking task for accessing the one or more storage racksfrom the storage area and placing the one or more storage racks at arack picking station, and a target transport task for transporting theone or more bins from the rack picking station to the picking station.11. The method of claim 10, wherein: the racking task for sendinginformation to a storage access system for controlling one or moreAutomated Guided Vehicles (AGVs) to transport the one or more storageracks to the rack picking station, the rack picking task includessending commands for operating a shelving robot to remove the one ormore bins from the one or more storage racks and placing the one or morebins on one end of a conveyor, and the target transport task is forsending commands to the conveyor to transport the one or more bins fromthe one end of the conveyor corresponding to the rack picking station toanother end of the conveyor that corresponds to the picking station. 12.The method of claim 8, wherein the operation includes: a bin transporttask for transporting the one or more bins from the picking station to arack feeding station, the racking task for placing the one or more binson one or more storage racks and placing the one or more bins and thecorresponding storage racks back in the storage area.
 13. A roboticsystem comprising: at least one processor; and at least one memorydevice connected to the at least one processor and having stored thereoninstructions executable by the processor for— identifying a set of twoor more available operations, each operation including a unique sequenceof two or more tasks that include at least a group manipulation taskand/or a racking task arranged adjacent to each other and/or adjacent toa unique task in comparison to other operations in the set; obtaining atarget condition representative of an end state of completing anoperation selected from the set of two or more available operations;based on the target condition, identifying a sequence of tasks from theset of two or more available operations; obtaining an access sequencebased on the identified sequence of tasks, wherein the access sequencerepresents an order for placing target storage containers at a firstsource location of a first task station; and implementing the identifiedsequence of tasks according to the unique sequence for (1) accessing atask target from one of the target storage containers at the firstsource location and transferring the task target to a destinationlocation at the first task station and (2) accessing the task targetfrom the second source location and for manipulating the task target incompleting the operation.
 14. The robotic system of claim 13, whereinthe at least one processor and the at least one memory device comprisesa master controller; and further comprising: a management systemoperably coupled to the master controller, the management systemconfigured to— identify an operation trigger for initiating an operationin the set of two or more available operations; and determine the targetcondition representative of an end state of completing the operation;and a storage access system operably coupled to the master controller,the storage access system configured to control the set of objecttransport robots or conveyors according to the timing to place the oneor more targets at the task stations and/or remove the one or more tasktargets at the task stations.
 15. The robotic system of claim 13,further comprising the manipulation robots configured to perform adevanning task, a sorting task, a storage grouping task, the groupmanipulation task, the racking task, a package opening task, a rackpicking task, a piece picking task, a target transport task, a bintransport task, or a combination thereof.
 16. A tangible, non-transitorycomputer readable medium having processor instructions stored thereonthat, when executed by one or more processors, cause the one or moreprocessors to perform a method, the method comprising: identifying a setof two or more available operations, each operation including a uniquesequence of two or more tasks that include at least a group manipulationtask and/or a racking task arranged adjacent to each other and/oradjacent to a unique task in comparison to other operations in the set;obtaining a target condition representative of an end state ofcompleting an operation selected from the set of two or more availableoperations; based on the target condition, identifying a sequence oftasks from the set of two or more available operations; obtaining anaccess sequence based on the identified sequence of tasks, wherein theaccess sequence represents an order for placing target storagecontainers at a first source location of a first task station; andimplementing the identified sequence of tasks according to the uniquesequence for (1) accessing a task target from one of the target storagecontainers at the first source location and transferring the task targetto a destination location at the first task station and (2) accessingthe task target from the second source location and for manipulating thetask target in completing the operation.
 17. The tangible, non-transientcomputer readable medium of claim 16, further comprising: identifyingincoming objects associated with a palletizing task of a correspondingoperation; computing packing simulations for grouping the incomingobjects according to one or more grouping conditions; setting apreparation factor for notifying the storage access system to preparethe target storage containers to receive the incoming objects; setting amove-in factor for notifying the storage access system of a first timingto place a pallet at the destination location; setting a move-out factorfor notifying the storage access system of a second timing to remove thepallet from the destination location; and wherein: obtaining the accesssequence includes receiving container information representative of thepallets prepared by the storage access system.
 18. The tangible,non-transitory computer readable medium of claim 16, wherein: obtainingthe access sequence includes receiving container informationrepresentative of the target storage containers prepared by the storageaccess system for a depalletization task of a corresponding operation,wherein the target storage containers are pallets having stored objectsthereon; further comprising: setting a move-in factor for notifying thestorage access system of a first timing to place a pallet at a sourcelocation in the task station; and setting a move-out factor fornotifying the storage access system of a second timing to remove thepallet from the source location.
 19. The tangible, non-transitorycomputer readable medium of claim 16, further comprising: identifyingincoming objects associated with a racking task of a correspondingoperation, wherein the incoming objects represent depalletized objectsor bins resulting from a picking task; obtaining available rackinformation representative of the target storage containers identifiedby the storage access system as candidates for receiving the incomingobjects for storage, wherein the target storage containers are storageracks configured to store the bins or the objects having items therein;setting a move-in factor for notifying the storage access system of afirst timing to place a rack at the destination location; setting amove-out factor for notifying the storage access system of a secondtiming to remove the rack from the destination location; wherein:obtaining the access sequence includes— computing rack sequences thateach represent a sequential combination of the identified target storagecontainers or a subset thereof; and identifying a selected sequence thatrepresents one of the sequential combinations selected by the storageaccess system.
 20. The tangible, non-transitory computer readable mediumof claim 16, wherein: obtaining the access sequence includes— receivingrack queue information representative of a sequence of storage rackshaving bins thereon that include ordered items; receiving picking queueinformation representative of a sequence of the bins, wherein the rackqueue information and the picking queue information represent sequencesderived by the storage access system based on an order; implementing theidentified sequence of tasks includes implementing a motion plan foroperating a shelving unit to remove a target bin from the storage rackand place the target bin on the destination location according to thepicking queue information, wherein the destination location represents aportion of a transport unit configured to transport the target bin fromthe task station to a subsequent station; further comprising: setting amove-in factor for notifying the storage access system of a first timingto place a storage rack at a source location in the task stationaccording to the rack queue information; setting a move-out factor fornotifying the storage access system of a second timing to remove thestorage rack from the source location; implementing a transport plan foroperating the transport unit to transport the target bin from the taskstation to an item source location of the subsequent station; andimplementing a subsequent task for picking one of the ordered items fromthe target bin at the item source location and placing the one of theordered items at an item destination location.